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Resumen 

Este artlculo presenta los resultados de investigaciones 
cuyo objetivo principal fue el de analizar las respuestas 
del gobiemo central guatemalteco frente a las necesidades 
lingiiistico educativas de los habitantes pertenecientes al 
grupo Garifuna de la Costa Atlantica de Guatemala. La 
situacion especifica de los Garifunas se estudio con el 
proposito de observar las respuestas de un Estado Pre- 
Modemo ffente a las aspiraciones de minorias nacionales 
en su seno. Tambien se analiza en estas lineas, el lugar 
ocupado por los Garifunas en relacion a los otros bloques 
lingiiisticos de guatemala, a saber, las poblaciones de 
habla Maya y Castellano. 
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Abstract 

This article presents the results of research carried out on 
the linguistic and educational needs of Guatemala's 
national minorities and the responses given to those needs 
by Guatemala's central government. Specifically, the case 
of the Garifuna population on the Atlantic Coast is studied 
with a view to understanding the educational policy 
dynamics underlying Pre-Modem States towards the 
multicultural groups' lack of participation and 
incorporation into the national polity. Lastly, the specific 
educational claims of the Garifunas are studied, 
particularly in relation to Guatemala's other linguistic 
blocks, namely, Mayan and Spanish language speakers. 



Introduccion 

La nocion contemporanea de Estado- Nacion vio sus origenes dentro del surgimiento del p 
No obstante, semejante concepcion del proyecto educative es incapaz de articular espacios 
Despiies de mas de un siglo de colonizacion, Francia todavia vacila entre acordar un estatu 
Sin embargo, la situacion de las minorias nacionales en el contexto de Estados cuyos patro 
EsiBtBisticulo analiza la situacion de uno de dichos Estados, en particular en referenda a los 
Para fines de esta discusion, en este trabajo se estudian los dos niveles de mayor concentra 



Sinopsis historica 

“La came de negro tambien se come.” (Nota 2) 

“Los cakchikeles nos vendieron a los espanoles y los negros no sufrieron 
nada durante la guerra. ^Porque los vamos a apoyar?” (Nota 3) 

La presencia Garifuna en Guatemala comienza a partir del siglo XVIII cuando gmpos de e 

Concentrados fundamentalmente en la zona de Izabal, departamento de clima calido-selvat 

Durante todo el periodo colonial y hasta bien entrado al siglo XX, la politica educativa gua 

En sintesis, los intereses corporativos de la Iglesia, de la oligarquia terrateniente y de los f 

Las politicas educativas -o su falta- permanecieron virtualmente inalteradas entre la Indepe 

En la actualidad, los indicadores sociales disponibles para el departamento de Izabal son d 

Desde principios del siglo XX hasta en la actualidad, la base economica de la region ha sid 

ipIMdipateGarifuna (tambien llamado Black Carib en algunas tipologias) pertenece a gmp 
en Guatemala, a saber, la poblacion maya hablante (entre 45 y 60 por cien de la poblacion 

En anos recientes, tambien han tenido lugar iniciativas de tipo educative en Honduras. Tal 

Por su parte, el sistema educative de Belize, pais que cuenta con 6 por ciento de poblacion 
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La respuesta del gobierno central guatemalteco a las necesidades educativas de la po 

hace muy poco tiempo, el Ministerio de Educacion o MINEDUC, no tem'a poHtica n 
mencionar la lentitud del proceso de reforma educativa que tiene que pasar por multiples n 

A nivel del pals en su conjunto, los Indices de financiamiento del sistema educativo refleja 

Esta escasez de recursos se traduce en la ausencia de programas de educacion bilingue a ni 

Otro elemento importante que contribuye a la situacion de paralisis educativa que vive el p 

A nivel operacional, tambien existen grandes limitantes para el desarrollo de una pedagogi 

Conclusiones: los limites de la reformas educativas en el contexto de Estados debiles 

“Creemos en una educacion para todos, alimentada por la vida, enriquecida por las alegria 

Desde su independencia de Espana hasta nuestros dlas, el Estado guatemalteco se ha caract 

Tras casi 40 anos de guerra civil, los Acuerdos de Paz firmados en 1 996 entre las fuerzas d 
Garifunas ganaran un espacio sociohistorico y un reconocimiento como actores politicos d 

Notas 

1 . Este trabajo se baso en investigaciones de campo conducidas en Guatemala y Hondu 

2. Respuesta de un oficial del ejercito guatemalteco a preguntas relativas a la incorpora 

3. Comentario de un llder indlgena durante im congreso de pollticas culturales celebrad 

4. Datos para este segmento se obtuvieron a partir del informe preparado por Naciones 

5. Entlendase en estas llneas, campanas de alfabetizacion en idioma castellano. Aimqu 

6. Por ejemplo, el documento preparado para UNESCO titulado Cultural policy in Gua 

7. Aunque una discusion a profundidad sobre el rol jugado por los establecimientos de 

8. Conversacion con una educadora Garlfuna residente en Ciudad de Guatemala. 
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Alexander v. Sandoval: 

A Setback for Civil Rights 

Kevin G. Weiner 
University of Colorado, Boulder 



Abstract 

This article confronts the serious implications of a recent 
U.S. Supreme Court decision, Alexander v. Sandoval, 
which eliminated an important legal avenue for civil 
rights plaintiffs. For over 35 years, individuals have been 
allowed to bring lawsuits directly challenging violations 
of rights set forth in the federal regulations implementing 
Title VI of the 1964 Civil Rights Act. Because these 
actions could be grounded in proof of disparate impact, 
rather than discriminatory intent, they allowed for some 
claims that could not go forward under other legal 
authorities, such as the Fourteenth Amendment. While the 
author concludes by identifying key remaining options, he 
highlights the real damage done by this decision. 
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I recently had occasion to remember a meeting three years ago 
with Richard Cohen, the legal director of the Southern Poverty Law 
Center in Montgomery, Alabama. At that meeting, he told me about 
his lawsuit on behalf of Martha Sandoval, a house cleaner from Mobile 
and a Mexican immigrant. Ms. Sandoval was denied a drivers' license 
because she could not pass the state's written exam. The voters of 
Alabama had passed an English-Only law, and the state interpreted 
that law to require that drivers' license exams be offered only in 
English (the only state with such a limitation). While Ms. Sandoval's 
working knowledge of English was sufficient to read road signs, it was 
not sufficient to take the exam. 

Mr. Cohen brought a class action lawsuit on behalf of Ms. 
Sandoval and the 24,000 other non-English speakers in Alabama, 
alleging that the state violated federal law by requiring applicants for 
drivers' licenses to take the written examination in English. The 
particular federal law that supported this lawsuit is known as Title VI 
of the 1964 Civil Rights Act (42 U.S.C. § 2000(d)). Title VI prohibits 
discrimination grounded in race, color or national origin. 

Like the Fourteenth Amendment's Equal Protection Clause, Title 
VI has been judicially interpreted to require proof of discriminatory 
intent. Proof limited to discriminatory effect, such as is clearly evident 
with the Alabama law, is insufficient. Yet, while courts have 
interpreted the statute itself to bar only intentional discrimination, 
federal regulations implementing Title VI, pursuant to § 602 of the 
statute, have been consistently given a broader interpretation (see 
regulations at 34 C.F.R. § 100.3(b)(2)). Lawsuits grounded in these 
implementing regulations are unique in that they allow people like Ms. 
Sandoval to make their arguments in federal court by showing the 
discriminatory effect (“disparate impact”) of a law. This brief article is 
about such Title VI disparate impact lawsuits and April's Supreme 
Court decision against Martha Sandoval, in Alexander v. Sandoval, 

121 S.Ct. 1511 (2001), eliminating the right of Ms. Sandoval and all 
others to pursue lawsuits directly enforcing the Title VI regulations. 

Intent versus effect — what's the difference? After all, the worst 
discrimination is surely intentional. The SPLC, for instance, has built 
an impressive record of court victories on behalf of victims of such 
egregious racism. These cases target the KKK and neo-Nazi 
organizations. The defendants are abhorrent, and the issue of racist 
intent cannot be seriously questioned. Further, we as a society do not 
want to encourage frivolous lawsuits grounded only in a statistically 
disproportionate effect on some minority group. What is the harm of 
limiting lawsuits to only those where discriminatory intent is clear? 

In a nutshell, policy makers today, no matter what their actual 
intent, are loath to expressly state an intent to discriminate. Even the 
English Only law that prompted Ms. Sandoval's lawsuit was likely 
promoted on facially neutral grounds such as unity, assimilation, and 
even fiscal efficiency. Within certain limits, policies that have a clear 
discriminatory impact should be closely scrutinized, and the 
government should have to offer reasonable justifications for them, 
even if there exists no smoking gun demonstrating an intent to 
discriminate. This is how courts approached Title VI disparate impact 
cases before April's Supreme Court decision. While the person 
bringing the case must prove that the practice in question has a 
disproportionate and negative impact on a protected group, the 
defendant (e.g., a state government or a school district) can then 
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respond by demonstrating a legitimate, nondiscriminatory reason for 
the practice (see Powell v. Ridge, 189 F.3d 387 (3d Cir. 1999); 34 
C.F.R. § 100.3(b)(2)). Frivolous lawsuits therefore fail either because 
of a lack of proof of disproportionate negative impact or because of an 
appropriate, nondiscriminatory reason for the practice. 

In Ms. Sandoval's case, the SPEC lawyers easily proved that the 
drivers' license rule had the prohibited effect of discriminating on the 
basis of national origin. Given that illiterate residents who could 
nonetheless understand spoken English were allowed to take the 
Alabama drivers' license exam in spoken form (with someone reading 
them the questions in English), the state could not justify denying 
residents like Ms. Sandoval the opportunity to take the written exam in 
a form that they could understand. The trial court agreed with Ms. 
Sandoval, as did the court of appeals. Normally, this would have been 
the end of the matter. 

But attorneys for the State of Alabama took one last shot. They 
asked the U.S. Supreme Court to declare that Congress never intended 
private individuals to be able to bring lawsuits directly under the 
authority of the Title VI implementing regulations. The Court agreed 

to hear the case, and on April 24, 2001 they reversed the judgment. By 
a 5-4 majority in Alexander v. Sandoval (the same breakdown of 
individual Justices deciding in favor of George W. Bush in Bush v. 
Gore), the Court concluded that Congress only intended these 
regulations to be directly enforceable by the Office of Civil Rights — a 
political body with very limited resources — ^not by a private right of 
action. 

The Court's decision even to consider this case was shocking. In 
the 37 years since the passage of the Civil Rights Act, the Court has 
several times given a tacit nod of approval to the now-forbidden 
private actions. Moreover, the issue of private actions brought under 
the Title VI regulations had been decided by 9 of the 12 U.S. Courts of 
Appeals, and there was no dispute; all agreed that such an action is 
legally appropriate. The Supreme Court will rarely hear a case 
addressing legal issues about which there exists no dispute among the 
various Courts of Appeals. Yet this Court reached out to hear the case. 

The fallout from Alexander is potentially enormous. For instance, 
in New York, ACLU attorneys may not be able to continue their Title 
VI action claiming that educational opportunities for the state's 
minority students are so inferior that they amount to discrimination. 
Similarly, in Pennsylvania, Philadelphia may have to dismiss its Title 
VI claim alleging that the Pennsylvania funding formula disparately 
impacts districts with higher minority enrollments. Dozens of other 
important civil rights cases will suffer a similar fate. 

For these reasons, the Court's decision in Alexander comes as a 
great disappointment. While expectations for the present Supreme 
Court may be low, courts as an institution play a crucial role in our 
constitutional system. American courts, particularly federal courts, 
once represented a refuge for children seeking access to educational 
opportunities. While the legislative and executive branches were 
responsive to those who sought policies expanding local control, 
pushing for tougher standards, or enhancing individual choice, the 
judicial branch served the interests of equity. Civil rights groups 
leveraged court mandates into broader, equity-minded educational 
policy reforms benefiting, among others, African Americans, Latinos, 
immigrants, and students with disabilities. Over the past two to three 
decades, litigation has undoubtedly been a less successful tool for 
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social justice. Yet this shift, partially attributable to a corresponding 
shift in judges' ideologies, need not be permanent; the judiciary retains 
its unique institutional position as protector of the constitutional rights 
of political minorities. 

The gloomy picture painted by the above description of 
Alexander and its probable aftermath should be tempered by the reality 
that, for better or worse, many judges' decisions in civil rights cases 
are grounded as much in their understanding of what is “fair” as in the 
specific elements of the legal claim for relief then at issue. From this 
perspective, what is important is that civil rights cases must find a 
legal toehold — some legislative justification to have the case 
considered. While the useful toehold provided by the implied right of 
action under the Title VI implementing regulations has now 
disappeared, other options remain. 

The most likely alternative course for future private actions may 
be offered by Section 1983 (of Title 42 of the U.S. Code), the 
reconstruction era legislation that authorizes lawsuits against the 
government or government officials responsible for the “deprivation of 
any rights, privileges, or immunities secured by the Constitution and 
laws.” The implementing regulations for Title VI may fall within the 
scope of Section 1983's protections. Actions brought under § 1983 
bypass the increasingly difficult implied right of action analysis. 
Congress expressly intended § 1983 to give civil rights plaintiffs 
access to the direct judicial relief. 

In fact, the Pennsylvania funding case mentioned above includes 
a disparate impact claim for relief, under the terms of Title Vi's 
implementing regulations, based on § 1983 {Powell v. Ridge, 189 F.3d 
at 400-403). See also, Bradford C. Mank, Using ^983 to Enforce Title 
VPs Section 602 Regulations, 49 U. Kans. L. Rev 321 (2001) (arguing 
that § 1983 should support private rights of action to enforce the 
disparate impact regulations issued pursuant to § 602). In the perhaps 
overly optimistic words of Justice Stevens (dissenting) in Alexander, 
“[T]his case is something of a sport. Litigants who in the future wish 
to enforce the Title VI regulations against state actors in all likelihood 

must only reference § 1983 to obtain relief.” 121 S.Ct. at 1527. 

Another alternative would be to turn to Congress for legislation 
that would return Title VI jurisprudence to its pre-Alexander state, as 
has been done with the Civil Rights Restoration Act of 1991 following 
the Supreme Court's decision in Wards Cove Packing Co. v. Atonio, 
490 U.S. 642 (1989). But such remedial legislation seems unlikely be 
approved by Congress or signed by the President in the near future. 

During my meeting with Richard Cohen of the SPLC, we 
discussed the importance of responding to systemic denials of 
educational rights with lawsuits that employ systemic legal 
approaches. \^en he argued Ms. Sandoval's case before the Supreme 
Court, he tried to protect one such systemic approach. The Supreme 
Court's decision to undermine Title VI unquestionably represents a 
severe setback for children seeking schooling opportunities. 

Eventually, the education rights community will be able to recover 
from this blow, but this will take time and the opportunity costs will be 
high. Instead of working to advance the cause of equal rights beyond 
its present state, advocates will have to devote their energies to 
repairing the damage incurred last week. In the meantime, many 
aggrieved students and others will find themselves without sufficient 
remedies. 

The Court's decision in Alexander was much more than a legal 
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abstraction; it marks a poignant shift in how Americans are allowed to 
treat one another. 
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Occupational Trends and Program Priorities 

Dan Rosenthal 
Auburn University 

Kitty C. Collier 

Alabama Commission on Higher Education 
Abstract 

Institutions of higher education that respond to the 
economic base in their region will remain competitive and 
be better positioned to obtain public funds and donor 
support. In addition to mandated program viability 
standards based on measures such as graduation rate, 
individual institutions and state coordinating boards can 
use ten-year occupational trend data to assess future 
program viability. We used an occupational demand 
model to determine whether academic programs can meet 
projected statewide needs for high demand and high 
growth occupations. For example, computer engineering, 
the highest growth rate occupation in Alabama, is 
projected to have 365 annual average job openings, with 
93.6% total growth over ten years. But only 46 computer 
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engineering majors graduate annually from all Alabama 
institutions of higher education. We recommend using an 
occupational demand model as a planning tool, decision- 
making tool, and catalyst for collaborative initiatives. 

Introduction 

Institutions of higher education operate in a highly competitive 
enviromnent. The push for increased state funds, new programs, more 
students, and expanded services led to increased emphasis on 
statewide coordination during the 1950s and 1960s as states sought to 
bring rationality to their rapidly growing higher education systems. 
However, competition took on new meaning during the 1980s and 
1 990s when state revenue for higher education began to dwindle or 
disappear as other state functions moved into priority funding 
positions. In addition, state legislatures and the public at large began to 
raise questions about accountability, performance, and productivity of 
the higher education systems (McGuinness, 1 997). At the same time, 
business and industry began calling for more effective responses to 
employment needs. 

By the late 1990s, it was clear that the market for higher 
education had changed. While the values and traditions of the academy 
remained "venerable sources of strength," institutions and their 
governing boards began to look to the external environment to 
understand the context in which those values and traditions must 
operate (Mingle, 1998). That environment included a changing labor 
market that demanded new skills for workers, the emergence of 
technologies such as the Internet, the challenge to market share of 
traditional colleges and universities by new providers of postsecondary 
education, and the intensely competitive and changing public policy 
context, which exacerbated cost, price, and productivity pressures on 
institutions of higher education (Mingle, 1998). 

To strike a balance between the demands of the market, the 
academy, and the public, some state-level higher education agencies 
have taken steps to link occupational trends to academic program 
priorities based on (a) the connection between higher education and 
the economy; (b) the current focus on meeting student and employer 
demands for job and skills training; (c) the need for public institutions 
of higher education to respond to state policy directives and 
demonstrate wise stewardship of public resources; and (d) the benefits 
of academic program planning and review in a statewide context. 

Connection Between Higher Education and the Economy 

In response to a growing demand for agricultural and technical 
education. Congress passed the Morrill Act of 1 862 to provide funds 
to establish land-grant colleges so that members of the working class 
could obtain a liberal, practical education. Every state and territory 
now has one or more land-grant colleges (National Association of 
State Universities and Land-Grant Colleges, 2000). 

The Carnegie Foundation for the Advancement of Teaching 
(1976) suggested the two best restraints on higher education are 
competition and state budgets. An institution that responds to the 
economic base in its region will remain competitive and be better 
positioned to obtain financial support from donors and legislators. 

According to Seymour (1988), one of the key characteristics of 
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strategic planning is "matching institutional capabilities with 
environmental conditions to achieve goals," and listed three 
considerations for determining program priority: mission, internal 
factors, and external factors. Toombs and Tierney (1991) recognized 
environmental factors, and specifically "market forces," in their 
components of curriculum design. 

Hines (1988) points out in a review of the relationship between higher 
education and state governments that: 

Increased investment of public funds in higher education 
toward the goal of increased economic development is 
predicated on the assumption that there will be a payoff, 
that economic activity will increase, that the tax base will 
expand, and that revenue will increase, (p. 33) 

Although it may be appealing to define mission, role, and 
program priorities in isolation, successful universities understand that 
this process cannot occur without consideration of their constituencies 
(Western Interstate Commission for Higher Education, 1 992a). In fact, 
many higher education plans include the education of persormel 
needed for "an advanced economy" (Western Interstate Commission 
for Higher Education, 1992b). For example, colleges and universities 
have added academic programs in areas such as computer engineering 
and management information systems when those knowledge areas 
became crucial for industrial development. 

Current Focus on Meeting Student and Employer Demands for 
Job and Skills Training 

Mingle (1998) noted that higher education is moving from a 
producer-dominated enterprise to one fully sensitive to and focused on 
the consumer. Public expectations of higher education appear to have 
no bounds, putting considerable pressure on colleges and universities: 

The American labor market is both extraordinarily diverse 
and exceptionally dynamic, making it difficult not only to 
generalize about the knowledge and "skill sets" college 
graduates need but also to make predictions about the 
foture demand for specific occupations. Through surveys 
and interviews of employers and external advisory groups, 
increasing numbers of colleges stay closely tuned to this 
changing job market. This information is shaping college 
programs in important ways. (p. 6) 

The Joint Commission on Accountability Reporting (1996) 
emphasized the need to stay focused on the consumer and recommends 
that institutions survey graduates and report placement rates ^p. 38- 
50). While placement is an important measure of accountability, it is 
more closely related to current employment than to future 
employability. Nor can placement identify employment possibilities 
for which no programs are in place. State-level coordinating agencies 
currently explore ways to conduct market analyses to determine how 
best to address the needs of their state. A review of the Alabama 
Commission on Higher Education (1999) recommended that the 
agency devote more effort and resources to statewide market analyses, 
and the State Higher Education Executive Officers Association offers 
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"State and System Tpols for Success in the New Market Environment" 
as an on-line seminar for state higher education agency staff. 

With respect to employer needs, there is a well-documented 
national disequilibrium between the supply and demand for 
information technology workers. Evidence for a severe worker 
shortage includes a high job vacancy rate, low unemployment, 
projected demand outstripping supply, higher than average salary 
increases, and demand for foreign workers (Freeman & Aspray, 1999). 
The national failure to develop sufficient technical talent is so severe 
that it could "substantially undermine" the future growth of the 
electronics and information technology industry (Platzer, Novak, & 
Kazmierczak, 1999, p 13). 

Need for Public Institutions of Higher Education to Respond to 
State Policy Directives and Demonstrate Wise Stewardship of 
Public Resources 

In recent years, many states have required academic program 
review and approval as a way to curb unnecessary duplication of 
programs among public institutions and to judge the appropriateness of 
existing programs (McGuinness, 1997). Most criteria for program 
review require employer needs analyses that indicate whether new or 
existing programs respond to employment needs. In some cases the 
link between employment opportunities and program graduates is a 
critical factor. For example, Alabama passed "program viability" 
legislation in 1996 that requires academic programs in all public 
institutions to meet minimum graduation rates or be terminated 
(Program Viability Act, 1996). After a three-year monitoring period of 
non-viable programs, institutions can request waivers for programs 
that still do not meet graduation rate standards provided they can 
document unique or extraordinary characteristics of the program. 
Factors that may be considered in this evaluation are placement of 
graduates in program-related areas of employment, success of program 
graduates, and market demands. Alabama institutions are evaluating 
how best to assess the link between graduates in low-producing 
programs and the state's employment needs. 

Benefits of Academic Program Planning and Review in a 
Statewide Context 

One economy driven process is the relationship between 
occupational trends and institutional programs. While individual 
institutions and groups of institutions can analyze occupational trends 
within their state, the institutional approach does not take into account 
what other in-state and out-of-state institutions are doing to meet the 
need. With limited resources available to higher education, 
institutional representatives, legislators, and policy makers must be 
committed to the most effective use of state dollars for the citizenry. A 
statewide approach to academic program planning and review requires 
institutions to think "outside the box," because what appears to be best 
for an individual institution may not be the best course of action for 
the region. While an institution may identify a high-demand 
occupation based on labor market projections and employer feedback, 
it must consider the productivity of existing and planned programs in 
the region to avoid potential duplication and market oversupply. The 
public trust requires that state dollars be spent on programs that have 
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high priority and provide substantial benefit. 

Previous Use of Occupational Trends at the State Level 

Although state-level agencies have been interested in links 
between occupational projections and academic programs for some 
time, the challenge has been to assess these relationships as a context 
for institutional program review. Some states have developed 
comprehensive proactive approaches to program needs assessment, 
while others simply react to institutional plans. 

Arizona. In 1998, the Arizona legislature challenged the Board 
of Regents and the State Board of Directors for Community Colleges 
to develop a mutual statewide process for identifying and meeting 
needs for advanced postsecondary education. In response, the two 
boards jointly convened the 1998 Higher Education Study Committee. 
The process utilizes a Joint Review Committee to evaluate requests for 
new or expanded programs on the basis of statewide criteria for need. 
Although needs assessment remained an institutional function, the case 
for a new program could be strengthened if multiple institutions 
partnership to meet the need. They recommend several sources of data 
to demonstrate program need, including the Arizona Department of 
Commerce, the Arizona Department of Economic Security, and the 
Bureau of Labor Statistics (Arizona Board of Regents and the State 
Board of Directors of Community Colleges for Arizona, 1998). 

Florida. Sanchez, Laanan, and Wiseley (1999) provide an 
excellent summary of state efforts to measure students' post-college 
earnings. Most initiatives follow program completers or graduates into 
the workplace to estimate average annual earnings or placement. 
Florida pioneered in this area with the Florida Education and Training 
Placement Information Program, established by a legislative directive 
and a joint agreement between the Florida State Department of 
Education and the Florida Department of Labor and Employment. 
Other states such as Ohio, California, North Carolina, Texas, and 
Washington have pursued similar approaches. However, these efforts 
provide little information on whether graduates are being trained in the 
fields most needed by employers. Idaho has taken a somewhat broader 
approach to needs assessment through statewide roundtable 
discussions and the use of specific advisory committees (Dodson, 
1999). 

Illinois. The Illinois Board of Higher Education is a member of 
a consortium with other state agencies committed to sharing labor 
market information. The board has conducted statewide analyses by 
field of study, comparing employment projections with graduate 
survey data. Typically, the board will conduct a statewide study of 
existing programs in a field, followed by institutional studies of related 
programs a few years later. The initial analysis gives institutions a 
usefol context for their own assessments. One recent board study 
included social work and human services (Illinois Board of Higher 
Education, 1997). 

A similar review of health professions education in Illinois in 
1992 compared projected average annual job openings with estimated 
total supply and number of degrees conferred in the state, and made 
recommendations for capacity adjustment in individual programs. The 
analysis was followed by recommendations for health professions 
education in 1993 and the implementation of policies for health 
professions education in 1995. The purpose of the study was to adjust 
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educational capacity, and the board recommended that some programs 
be reduced and monitored, some be maintained, and some be expanded 
(Illinois Board of Higher Education, 1995). 

In 1998, the board published a report that identified and 
proposed solutions to meet the educational needs in Lake County 
(north of Chicago). The study included market research conducted by 
a private consulting firm. The board staff convened a number of 
forums to provide an opportunity for Lake County residents to express 
their educational needs, and conducted further research to analyze 
demographic and economic data relevant to educational demand and 
need. They used the number and percent of positions in Lake County 
that required postsecondary education as compiled by the Illinois 
Occupational Information Coordinating Committee to assess 
educational demand (Illinois Board of Higher Education, 1998). Based 
on the results, the board established a University Center in Lake 
County that offered high quality, convenient, and affordable education 
built on the resources and programs of existing institutions. 

Ohio. Gottlieb (1995) used an industry-occupation matrix 
combined with occupational projections to identify industries likely to 
provide future entry level and advanced training jobs as a way to re- 
prioritize job training programs in two-year institutions in the 
Cleveland-Akron area of Ohio. 

Wisconsin. The University of Wisconsin System supports a 
market research unit that works with universities to identify needed 
programs in their region by looking at demand from employers and 
students. Faculty still identify areas of interest for new programs, but 
the market research unit then samples regional businesses using the 
Dunn and Bradstreet list (Sell, 1999). 

Statement of the Problem 

The state of Alabama needs a systematic statewide process for 
comparing occupational projections with the number of graduates of 
academic programs for use in program planning. Although individual 
institutions have made such comparisons as needed to foster strategic 
planning for program prioritization, resource allocation, curriculum 
development, and course availability, the need to analyze occupational 
and graduation data at the state level has been heightened by several 
recent developments. They include more limited resources to support 
higher education, passage of a program viability bill with provisions 
for waiver of non- viability based on factors related to meeting 
occupational needs, and recommendations by the Evaluation 
Committee of the Alabama Commission on Higher Education to 
increase the agency's use of market research as a planning tool. 

The purpose of this study is to compare occupational projections for 
the state of Alabama with graduation rates in corresponding academic 
programs to provide a context for state and institutional policy 
decisions on current programs and new program initiatives, and to 
comply with recent program viability legislation. 

Methods 

We employed three major tools to establish a context for state 
and institutional policy decisions: (a) statewide employment 
projections, (b) number of degrees conferred, and (c) a crosswalk to 
relate one with the other. We limited the analysis to high-dememd and 
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fast-growing occupations in Alabama that require a Bachelor's degree 
or higher, as identified by the Alabama Department of Industrial 
Relations. They define high-demand occupations as having at least 
535 average annual job openings. Fast-growing occupations have at 
least 50 average annual job openings and an average annual growth 
rate of at least 3.2% (Alabama Department of Industrial Relations, 

1998) . 

Employment Projections 

The Bureau of Labor Statistics has prepared national 
employment projections since 1957 (U. S. Department of Labor, 

1995). Minimal input data was available at first, but by the early 1970s 
a standard methodology was developed that is still in use today (U. S. 
Department of Labor, 1986; 1997). The bureau releases ten-year 
national employment projections every other year. It uses many factors 
to make projections, including the composition of the labor force, 
economic growth, demand, and occupational trends. For example, 
occupational trends are based on data collected from an Occupational 
Employment Survey prepared and summarized by the bureau. The 
survey is administered by each state, and contains data on 
approximately 775 occupations in 350 industries. The data includes 
number of employees and salary range by occupation, providing 
regular empirical information on occupational employment. 

Information is stored in a projections database that is 
programmed to generate employment trends over a ten-year period. 
The bureau makes several key assumptions during the projection 
process. For example, work patterns will not change during the 
projection period (length of average work week), broad social and 
educational trends will continue, there will be no major war, there will 
not be a significant change in the size of the armed forces, and there 
will be fluctuations in economic activity due to the business cycle. The 
most recent national projections localized for the state level are for the 
ten-year period 1996 - 2006 (Silvestri, 1997). (See, also, U. S. 
Department of Labor, 1998.) 

The bureau monitors and validates projections, and exceptions 
to general assumptions are reported. For example, they found that both 
the manufacturing and health industries suffered unexpected setbacks 
in 1 998 that were attributed to the Asian economic crisis and more 
stringent health care reimbursement policies (Goodman & Consedine, 

1999) . 

The bureau conducted a detailed analysis of the educational 
requirements of occupations and published the minimum amount of 
preparation that most employers required. However, requirements can 
vary from employer to employer, and there may be more than one way 
to qualify. For example, the educational preparation listed for 
registered nurses is associate degree, although baccalaureate graduates 
take the same licensure exam and are hired for the same entry-level 
positions. For that reason, bureau educational requirements for each 
occupation must be evaluated for accuracy in a given state (U. S. 
Department of Labor, 1995; 1996). 

The demand for college graduates continues to increase as duties 
become more complex due to new technology and changing business 
practices. This phenomenon, called educational upgrading, accounted 
for one-third of the college-level jobs created between 1983 and 1994 
(Shelley, 1996). Changes in employment growth can be due to the 
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growth of an industry as well as changes in occupational structure. For 
example, employment in the health-related professions is expected to 
increase along with growth in the health services industry. More use of 
computer technology, a structural change, will accelerate the need for 
systems analysts and programmers, and reduce the need for typists 
(Franklin, 1997). 

Nationally, the ten fastest growing occupations that require a 
bachelor's degree are: (a) database administrators, computer support 
specialists and all other computer scientists, (b) computer engineers, 

(c) systems analysts, (d) physical therapists, (e) occupational 
therapists, (f) special education teachers, (g) speech- language 
pathologists and audiologists, (h) physician assistants, (i) residential 
counselors, and (j) securities and financial services sales workers (see 
Table 1); (U. S. Department of Labor, 1998, p. 52). 

Table 1 

Fast-Growing Occupations in Nation 
Requiring a Bachelor's Degree, 1996-2006 



Occupation 


Ten-Year 
% Growth 


Database Administrators 


118 


Computer Engineers 


109 


Systems Analysts 


103 


Physical Therapists 


71 


Occupational Therapists 


66 


Special Education Teachers 


59 


Speech-Language Pathologists 
and Audiologists 


51 


Physician Assistants 


47 


Residential Counselors 


41 


Financial Services and Sales 


38 



The bureau provides each state with a data set for making local 
projections. Using special software, states prepare projections that are 
parallel to the national but based on local populations, industries, and 
employees. We used the Alabama Occupational Trends data for April 
1998, which are localized from federal projections, to estimate 
statewide employment demand in various occupations (Alabama 
Department of Industrial Relations, 1998). We defined employment or 
occupational demand as the projected annual average number of job 
openings in Alabama for the period 1 996 -2006. Specifically, we 
evaluated the projected employment need for all high-demand and 
fast-growing occupations that require a bachelor's degree or higher (we 
excluded first professional preparation). In Alabama these occupations 
are: (a) secondary school teachers, (b) general managers and top 
executives, (c) registered nurses, (d) elementary school teachers, (d) 
systems analysts, (e) special education teachers, (f) accountants and 
auditors, (g) computer engineers, (h) engineering, math and natural 
science managers, (i) residential counselors, (j) preschool and 
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kindergarten teachers (combined group), (k) physical therapists, (1) 
operations research analysts, (m) speech- language pathologists and 
audiologists, and (n) occupational therapists. 

Number of degrees conferred 

Public and private institutions of higher education in Alabama 
prepare a mandatory completions survey as one of the federal reports 
used in the Integrated Postsecondary Education Data System of the 
National Center for Education Statistics (U. S. Department of 
Education, 1994 - 98). The completions survey is a comprehensive 
report of graduates organized by award level and curriculum. The 
curriculum area is designated by a program description and six-digit 
code based on the national Classification of Instructional Programs 
taxonomy. (For more information on academic program definitions, 
see Morgan, Hunt, & Carpenter, 1991). Institutions forward an annual 
completions report to the Alabama Commission on Higher Education, 
the statutory state coordinating agency, which maintains a longitudinal 
statewide repository of these reports (Alabama Commission on Higher 
Education, 1994 - 98). 

Using this curricula completion information we were able to 
determine the number of degrees conferred in a given program in a 
given year in Alabama. For example, the number of completions in 
registered nurse preparation programs is the sum of the number Of 
nursing degree completions reported under program code 51.1601 at 
each institution in a given year. We can use this method to determine 
the total number of degree completions reported for any academic 
discipline in the state. In this study, we define degrees conferred as the 
average annual number of completions reported by postsecondary 
institutions in Alabarria based on the five-year period 1993-94 through 
1997-98 (July 1 - June 30 reporting period). Averages include public 
and private institutions and are based on Integrated Postsecondary 
Education Data System reports. 

Crosswalk 

Some occupations listed in the state employment projections 
have an obvious relationship to an instructional program reported in 
the completions survey. When questions arose, we consulted a 
crosswalk database to help identify the relationship. The database 
relates occupations to academic programs by linking an occupational 
employment survey code to an instructional program code (National 
Crosswalk Data Center, April, 1999). 

For example, based on statewide repository data and prior 
knowledge, we identified 24 Alabama colleges and universities that 
report baccalaureate and master's degree completions in programs that 
lead to employment in the occupational category systems analyst. 
Colleges confer degrees in the following related instructional programs 
(and program codes): (a) computer and information sciences, general 
(1 1 .0101), (b) information sciences and systems (1 1 .0401), (c) 
computer science (1 1.0701), (d) computer and information sciences, 
other (1 1.9999), and (e) management information systems and 
business data processing, general (52.1201). Note that all of these 
programs are offered at the bachelor's level, and programs (a) and (e) 
are offered at the master's level as well. 

A crosswalk database query for systems analyst degree program 
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codes pointed to the following occupations (and occupational codes): 

(a) systems analysts, electronic data processing (25102), (b) data base 
administrators (25103), (c) computer support specialists (2M04), (d) 
computer programmers (25105), (e) computer programmer aides 
(25108), (f) all other computer scientists (25199), and (g) computer 
science teachers, postsecondary (31226). 

The crosswalk query shows that graduates who earn a systems 
analyst or related degree in college are reported on the Occupational 
Employment Survey as working as systems analysts, as well as in a 
cluster of related jobs. Thus, we can link the number of systems 
analyst and related degrees conferred to the number of projected job 
openings for systems analysts and related occupations, although some 
graduates will enter other fields. Note that to be conservative in our 
estimate of needed graduates, we limited the number of projected job 
openings to systems analyst, eliminating all of the related fields. The 
articulation between academic program and occupation will be more 
precise for some occupations than others. Occasionally, crosswalk 
relationships were adjusted to better reflect specific conditions in 
Alabama. 

Findings 

The application of this model to 1 5 high-demand and fast- 
growing occupations requiring a minimum of a bachelor's degree 
yielded the general conclusion that existing programs in Alabama 
colleges and universities will supply a sufficient number of graduates 
to meet the state's demand for many of these occupations through the 
year 2006. For reporting purposes, we grouped the results of 15 
occupational demand analyses into three categories: (a) occupations 
where the supply of graduates is projected to meet or exceed demand, 

(b) occupations where the supply of graduates is projected to be 
insufficient to meet demand, and (c) occupations requiring further 
study. 

Occupations Where the Supply of Graduates is Projected to Meet 
or Exceed Demand 

The supply of graduates is projected to meet or exceed the 
demand for (a) general managers and top executives, (b) registered 
nurses, (c) elementary school teachers, (d) accountants and auditors, 
(e) engineering, math and natural science managers, (f) residential 
counselors, (g) preschool and kindergarten teachers, (h) physical 
therapists, (i) speech-language pathologists and audiologists, and (j) 
occupational therapists. 
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Thousands 



Figure 1. High-demand and fast growing occupations where the 
supply college graduates is projected to meet or exceed statewide 
need. 

Occupations Where the Supply of Graduates is Projected to be 
Insufficient to Meet Demand 

The supply of graduates is projected to be insufficient to meet 
the demand for (a) systems analysts, (b) special education teachers, (c) 
operations research analysts, and (d) computer engineers. 
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Figure 2. High-demand and fast growing occupations where the 
supply college graduates is projected to be insufficient to meet 
statewide need. 



Occupations that Require Further Study 

The supply of graduates and demand for secondary school 
teachers requires further analysis with respect to need in specific 
certification areas. 



tittTv//ftnnn n«5ii f»rlii/p.naa/vQn9S/ 



29 



10/29/01 





EPAA Vol. 9 No. 25 Rosenthal & Collier: Occupational Trends and Program Priorities Page 12 of 19 




0 500 1000 1500 2000 2500 3000 

Actual Number 



Figure 3. High-demand and fast growing occupations that require 
further study. 



Discussion 

We recommend three primary uses for an occupational demand 
model: (a) as a planning tool, (b) as a decision making tool, and (c) as 
a catalyst for collaborative initiatives. 

Planning Tool 

A model of occupational demand provides a valuable contextual 
base for statewide discussions of employment needs, and ways that 
higher education can address those needs. Although a demand model 
cannot provide absolute judgments on the need for particular 
programs, it can provide a starting point for asking the right questions. 
For example, we found that Integrated Postsecondary Education Data 
Systems completions in secondary education are not the best source 
for the available supply of teachers. Institutions can award teaching 
certificates without offering academic programs, and teachers may be 
certified through alternative routes. Therefore, degree completions 
surveys may underestimate the total number of certifications awarded. 
The Oklahoma State Regents commissioned the Southern Regional 
Education Board (1998) to conduct a study of educator supply and 
demand by type of certification. In Alabama, consultation with 
officials at the State Department of Education suggested that for the 
most part, Alabama produces more new teachers than local education 
agencies need, with the exception of areas such as special education, 
foreign languages education, and sciences other than biology. Given 
the difficulty of hiring foreign language teachers and the low 
productivity in many foreign language programs in the state, we need 
to formulate policies that lead to an understanding occupational needs 
and focus on solutions. State policy formulation should involve all 
stakeholders in meaningful deliberations (institutional representatives, 
the state coordinating board, the state department of education. 
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business leaders, legislators, etc.). 

Decision Making Tool 

Individual institutions and state coordinating boards can use data 
based on an occupational demand model as a tool in making academic 
program decisions. The relationship between number of college 
graduates and occupational demand can serve as an important source 
of information for determining whether institutions of higher 
education are meeting the employee training needs of business and 
industry. If an occupation is identified as high-demand or fast- 
growing, and an institution's faculty express interest in developing an 
academic program in this field, they should consider the productivity 
of existing programs, and the potential productivity of newly approved 
programs. Several years ago the Alabama Commission on Higher 
Education approved three new master's level programs in physical 
therapy. When the new programs were included in estimates of future 
productivity, the supply and demand for physical therapists in the state 
was in approximate balance, even though physical therapy is projected 
to be a fast-growing occupation during the period 1996-2006. 
Institutions will be better able to allocate limited resources to 
appropriate programs when the regional productivity of existing 
programs is considered. 

We view comparisons of occupational projections with 
academic program graduates as a focal point for discussion, rather than 
an absolute measure of need to continue existing programs or establish 
new ones. A complex decision, such as whether or not to close an 
academic program, requires broad- based judgments that include 
multiple components in the decision process, such as job placement of 
current students, emerging market trends, and research support 
(particularly at the graduate level). While research on occupational 
trends is an important information source, we view it as part of a larger 
decision-making framework. 

Institutions can use the model to identify areas that are not 
currently being addressed by the educational system. For example, 
information technology (computer engineers, systems analysts) is an 
area where existing programs are not producing adequate numbers of 
professionals. Institutions may want to implement strategies to 
increase enrollment in existing programs or plan new ones. Another 
useful process is to identify high-demand and fast-growing programs 
that are not offered by any institution in the state. Finally, while the 
selection of an occupation is an individual choice, educational 
organizations can help consumers make informed decisions by 
providing valid information about the prospects for occupational 
employability. 

Catalyst for Statewide Cooperative Initiatives 

It is difficult for competing institutions to foster cooperative 
ventures, and collaboration is not the norm among institutions of 
higher education. However, an occupational demand model can 
identify program areas that are ripe for cooperative initiatives. 
Relationships can be encouraged through collaborative inter- 
institutional discussions and financial incentives, and cooperative 
programs can be established that benefit the state as a whole. 
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Other Influences 

We used an occupational demand model to compare projected 
employment needs with statewide graduation rates as a metric for 
program resource allocation. We mentioned other influences on the 
demand model, such as the goodness of fit between occupations and 
academic degrees, variations in minimum educational job 
qualifications, migration of graduates to (and from) other states. In 
Alabama, there are graduates of out-of-state corporations that are not 
accountable to the Alabama Commission on Higher Education. These 
influences argue for using an occupational demand model as part of a 
broader decision-making process. 

Notes 

This article is based on a presentation at the 39th Annual Forum 
of the Association of Institutional Research, Seattle, Washington, June 
2, 1999. We wish to thank Douglas Dyer, Chief, Labor Market 
Information Division, Alabama Department of Industrial Relations, 
and his staff, for providing us with state employment projections and 
related materials, and for meeting with us to discuss this project. 
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Negotiated Learning: Union Contracts and Teacher 
Professional Development 

Paul V. Bredeson 
University of Wisconsin-Madison 

Abstract 

In this article, I report the results of an investigation that 
examined the impact of teacher union contracts on the 
development of professional learning communities in 
schools. There are three primary sources of data used in 
the study: 1) 100 written teacher union contract 
documents; 2) structured interview data from 21 educators 
(school superintendents, principals, directors of staff 
development, and teacher union representatives; and 3) 
focus group interview data from educational leaders in 
schools. The analysis and discussion focus on five areas 
related to teacher professional development with 
implications for policy and practice: explicit language 
covering opportunities for teaching learning in their work; 
governance and decision making structures, that is, 
specific provisions covering wages, hours, and conditions 
of employment; the description of legitimate and 
sponsored activities for the professional development of 
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teachers; and the resources supporting the on-going 
professional growth of teachers. The findings indicate that 
rethinking, restructuring, and organizational re-culturing 
in schools are initial expressions of a new unionism that 
has the potential to lead to the development of more 
powerful professional learning communities in schools. 

Introduction 

Among educational policy makers, researchers, and 
practitioners, there is an emerging consensus that teacher professional 
development is vitally important to educational reform as we approach 
the next millennium. In fact, it seems trite to assert that teacher 
professional development is critically important to school 
improvement focussed on enhanced student learning outcomes. 
Nevertheless, there continues to be a need to communicate the 
importance of continuous learning and development for educators, 
individually and collectively, to people in and out of schools. Without 
clearly articulated and documented evidence of its overall contribution 
to school success, professional development can easily become the 
victim of capricious budget cutting, or worse, be relegated to the scrap 
heap of educational fads and ephemeral educational elixirs. 

The link between teacher professional development and union 
contracts is one that has been forged over decades of collective 
bargaining between teachers' associations and local school boards. 
“After all, unions are potentially powerful collaborators because they 
negotiate the allocation of time in school and define a teacher's official 
duty day and psychological work role relationships” (Kerchner, 
Koppich, & Weeres, 1997 p. 173). In additional to traditional areas of 
bargaining (wages, hours, and conditions of employment), recent 
school reforms and new political realities have forced teachers and 
school boards to re-examine their contractual relationships. 

Though there are many dimensions of teacher union activities 
supporting teacher learning in classrooms, schools, and beyond, this 
study focuses only on written teacher contracts and their 
administration. I was particularly interested in knowing if the language 
in teacher union contracts stated explicitly, or reflected indirectly, the 
importance that schools, administrators, and teachers placed on 
professional development. The purpose of this study was to examine 
teacher union contracts and the impact of these agreements on teacher 
learning. The following questions guided the study. First, to what 
degree is teacher professional development explicitly addressed in the 
language of local collective bargaining agreements between school 
boards and teachers' unions? Second, in what way(s), if any, does 
contract language covering wages, hours, and conditions of 
employment influence teaching learning and teachers' capacity to 
improve their practice? Third, according to teachers and 
administrators, what aspects of contracts and their administration 
affect teacher learning and professional growth? 

Background 

Teacher Professional Development 

Even the casual reader of educational reform reports, legislative 
mandates, and contemporary educational literature would soon 
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discover one common theme — ^teacher professional development is 
critical to systemic educational reform and school improvement 
focussed on enhancing learning outcomes for all children in public 
education. These include calls to: create stable, high quality sources of 
professional development for teachers (What matters most: Teaching 
for America's future, 199S)\ incorporate teachers' learning into the 
fabric of teachers' daily life (Bredeson, in press; Tomorrow's Schools 
of Education, 1995; Teachers take charge of their learning, 1996); 
establish professional development as a central component of state and 
local educational reform (Houghton & Goren, 1995; Darling- 
Hammond & Sykes, 1999; Johansson & Bredeson, 1999); transform 
professional development to meet urgent edueational needs (Corcoran, 
1995; Porter, Smithson, & Osthoff, 1994); consider alternatives to 
traditional training models of staff development (Little, 1993; Sparks, 
1994); deal more directly with issues of racism and inequity in sehools 
(Weissglass, 1997); develop practices that support new conceptions of 
teaching, learning, and schooling (Lieberman, 1995; Loucks- Horsley, 
Hewson, Love, & Stiles, 1999; Hawley and Valli, 1999); effect 
behavioral change and improved practice (Osterman and Kottkamp, 
1993; Guskey, 1995); and break the mold to classroom practices 
through new professional development practices (McLaughlin & 
Oberman, 1996). 

There is a large body of evidence that identifies design 
principles for effective, high quality professional development. 
Developing guidelines for the design, delivery, and evaluation of 
outcomes is an important first step in the development of professional 
learning cultures in schools. Examples of these guidelines can be 
found in the Standards for Staff Development (NSDC, 1995; AFT, 
1995; Darling-Hammond & McLaughlin, 1995; and Darling- 
Hammond & Sykes, 1999). The National Partnership for Excellence 
and Accountability in Teaching (NPEAT, 1998), for example, 
identified eight design principles based on current research and best 
practices in schools. The most effective professional development: 

1 . Focuses on analyses of student learning, especially the 
examination of differences between actual student learning 
outcomes and goals and standards for student learning 

2. Involves teachers identifying their own needs and developing 
learning experiences to meet those needs 

3. Is school-based and embedded in teaehers' daily work 

4. Is organized around collaborative problem-solving 

5. Is continuous and on-going with follow-up and support for 
further learning 

6. Incorporates evaluation of multiple sources of data detailing 
student learning and teacher instructional practices 

7. Provides opportunities for teacher to link the theory that 
underlies Imowledge and skills they are learning 

8. Is connected to a eomprehensive ehange proeess focused on 
improved student learning. (NPEAT, 1 998) 

Developing lists of design principles is important, but 
identifying them is generally much easier than implementing them 
effectively. The hard work comes in putting the design principles into 
practice with real people in the dynamic and complex environments of 
sehools. Teacher union contracts provide an important lens for 
examining the organizational structures and dynamics of teacher 
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professional development and work. 

Defining the concept of professional development 

The term professional development, ubiquitous in current 
literature, is often used interchangeably with such terms as staff 
development, in-service, skills training, and continuing education. I 
believe there are meaningful distinctions among these terms as well as 
conceptual limitations. To avoid confusion and to clarify the concept 
of teacher professional development, I have developed a definition 
grounded in research and current literature cited above. Professional 
development refers to learning opportunities that engage teachers' 
creative and reflective capacities to strengthen their practice. In this 
conceptualization, my intention is to highlight three critical 
dimensions of professional development. First, professional 
development has to do with learning opportunities. These may be 
formal or informal, individual or group, and be delivered in dozens of 
different ways. The important dimension, often assumed but not 
explicitly stated by many writers and practitioners, is that learning, not 
the activity, is the focus of professional development experiences. 
Thus, learning opportunities are not narrowly limited to discrete 
activities, events, or days on the school calendar. Second, if learning 
opportunities are designed to make a difference in the way(s) teachers 
think about their work and practice what they know, the learning 
opportunities must engage teachers' creative and reflective capacities. 
By this I mean these learning opportunities tap into teachers' natural 
inclination to reflect on, personalize, and transform new knowledge 
and skills in ways that fit their personal style as well as the context of 
their work. Osterman and Kottkamp (1993) describe the relationship 
between reflection and professional development. “Reflective practice 
is viewed as a means by which practitioners can develop a greater 
level of self-awareness about the nature and impact of their 
performance, an awareness that creates opportunities for professional 
growth and developme nt” (p. 19). The third component of this 
definition is to strengthen teachers' practice. Billions of dollars are 
f spent each year on professional development in the United States 

(NCTAF, 1996). This investment is made primarily because taxpayers, 
policy makers, and practitioners believe learning opportunities that 
engage teachers' creative and reflective capacities will deepen teachers' 
understanding of their work and ultimately lead to improved teaching 
practices that benefit children in schools. 

Teacher Unions and Professional Development 

Teaching has become the most unionized occupation in the 
United States, and local contracts now create a complex systems of 
rules that r egulate labor-management relations ’’(Sykes, 1999, p. 240). 

The legacy of industrial unionism. As the size of schools and 
school districts in the United States grew over the past century and 
half, pi imarily as the result of massive consolidation of school 
districts, it seemed only natural that the education sector would look to 
othpr sectors, business and industry in particular, for organizational 
models and principles that could be used in managing increasingly 
complex school systems. Based on principles of scientific 
management, educational decision making became much more 
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centralized with, “power and authority accrued to school district 
headquarters (and, not incidentally, was lodged firmly in the hands of 
administrators)” (Koppich and Kerchner, 1999, pp. 317-318). So it 
was only natural that as teachers experienced and began to examine 
their formal working relationships with local school districts, they too 
looked to industrial examples for guidance. “Thus, both the AFT and 
NEA modeled their operation on the unions that had served American 
factory workers so well in the post- World War II period” (p. 317). 
Early on in the developing relationship among teachers, school boards, 
administrators the parties met and conferred on issues of interest to 
teachers in what Kerchner and Mitchell (1988) characterized as first 
generation unionism. From this first generation of unionism, we now 
have 34 states with collective bargaining laws that govern the 
relationship between teachers and their school districts. By the late 
1950s the formal relationship between teachers and school districts 
entered a second generation of unionism steeped in “good faith” 
collective bargaining where wages, hours, and conditions of 
employment became the focus of teachers' interests through the \vritten 
contract and management (e.g., school boards and administrators) 
retained control of policy and operational decisions in education. “This 
presumed bifurcation of union-management interests is reinforced by 
the statutorily restricted scope of bargaining. State laws define those 
issues about which union and management can bargaining and those 
that are excluded from negotiations” (Koppich and Kerchner, p. 318). 

Various change forces and challenges in education over the past 
half century moved teachers and school districts from first generation 
unionism to second generation unionism characterized by distributive 
negotiations where, “Bargaining is about dividing up the spoils — 
money, rights, power — and carrying them away” (p. 319). Recently, 
educational reform initiatives accompanied by increasing demands for 
school district/teacher accountability for student learning outcomes 
have moved teachers' unions and school districts to rethink the 
traditional boundaries on their working relationships codified in 
collective bargaining agreements. In addition there are a number of 
exciting, substantive changes in teacher education and professional 
development that challenge teacher unions, administrators, and local 
school districts to rethink their relationship to professional 
development (Kerchner et al., 1997). 

Linda Darling-Hammond (1998) argues for research that more 
closely examines connections between educational reform and teacher 
professional development. “To build lasting support for change, 
research about successful professional development initiatives needs to 
be translated into policies that will penetrate widely and 
comprehensively. These would include policies that influence school 
finance, salaries and incentives, preparation, recruitment, and retention 
of well- qualified teachers” (p. 13). Most likely, the translation of this 
research will be formalized in policies and practices that are closest to 
teachers and their work. These clearly include local collective 
bargaining agreements between school boards and teachers unions as 
well as a wide variety of side agreements, school/policy manuals, and 
other written documents governing these relationships. 

New Unionism. So what does this new unionism look like? To 
begin, there is substantial evidence that teacher unions have long been 
involved in socializing and supporting teachers in local school district. 
“Teachers' organizations participate in teacher socialization through a 
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variety of means. First, they help set many of the terms for teachers' 
work and learning in the larger district through collective bargaining, 
including the scope of legitimate teaching activities within and beyond 
the school day, the nature of and expectations for leadership positions, 
participation in decision-making, and opportunities for professional 
development” (Bascia, 1999, p. 12). She makes the case that in school 
systems where teachers do not receive sufficient support for their 
teaching, teachers' organizations through a wide variety of supporting 
activities and structures are, “increasingly are filling in the gaps 
resulting from educational policies that assume unrealistically 
simplistic, technical vies of teaching and policy implementation” (p.3). 

More formally, there are at least three general strategies 
teachers’ unions and school districts have employed to move toward 
more collaborative bargaining in which unions and management are 
seeking common ground to deal with issues of mutual interest and 
benefit. “The parties treat each other as professionals and consciously 
consider the issues that are important to both and the trade-offs each 
side can accept. It is this conception of negotiations that has given rise 
to locally based union reforms” (Koppich and Kerchner, 1999, p. 319). 
These include: 1) joint committees that, “expand the portfolio of the 
negotiated agreement and move substantive discussions of education 
policy and practice beyond the legally restricted scope of bargaining;” 
2) trust agreements, “legally binding bilateral accords that sit outside 
the collectively bargained contract;” and 3) waivers, specific 
provisions or requests that allow school districts and teachers' unions 
to request relief from specific provisions or jparts of the existing 
collective bargaining agreement (p. 320). 

Despite the promise of these locally based efforts, Koppich and 
Kerchner (1999) view these as mere tinkering at the margins of 
traditional unionism that, “no matter how faithfully conducted and 
thoughtfully executed, have failed to move unions and districts much 
beyond the education reform starting gate” (p. 321). They argue that, 
“Teacher unions have organized teachers' economic lives and brought 
stability to working conditions. Now they have an opportunity to lead 
the transformation of education by embracing a new set of first 
principles of unionism; organizing around quality, organizing around 
schools, and organizing a flexible teacher labor market” (p. 321). 

Though there are a number of positive aspects of the new spirit 
of unionism around issues of educational reform and teacher learning 
and growth, there are critics especially when the results in public 
policy tend to be limited to bilateral agreements between teacher 
unions and school districts. For example, Cibulka (1999) points out 
how conservative critics argue that teacher unions already have an 
inordinate amount of influence in schools and that their highly vested 
special interests may turn negotiated policies and agreements into 
documents that, “run public schools for their own benefit and inculcate 
their own values” (p. 173). Joseph Murphy (1999) describes the impact 
of new unionism and compacts on consumers of public education. 
“Public sector unions in particular are key instruments in the growth of 
bureaus and concomitant subordination of consumer interests to the 
objectives of the employees themselves. Ramsey (1987) concludes that 
when the economic influence of unions is combined with political 
muscle, public sector unions have considerable 'ability to tax the rest 
of society'[p. 97]” (p. 41 1). Finally, Joel Spring (1993) advises caution 
in the expansion of language in teacher union contracts to include such 
non-economic policy matters as professional development. He argues 
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that expanding union contract language into such areas as professional 
development may have unintended negative consequences. For 
example, union influence in noneconomic areas often reduces public 
control, limits administrator influence (especially that of principals), 
results in overly formal and complex governance and practices around 
teacher development, and may negatively influence district and school 
decisions about resource allocations and educational policy by 
supporting the interests of teachers over those of students and the 
community. 

Methods 

Data collection 

To address the research questions, I collected and examined 
three sources of data: 1) written collective bargaining agreements; 2) 
interview data (n=2 1 ) from superintendents, principals, directors of 
staff development, state teacher association administrators, and teacher 
union presidents; and 3) focus group interview data. 

Collective Bargaining Agreements. There are 427 local school 
districts in the state, each with a negotiated master agreement between 
the local school board and teachers' association. Teachers are 
represented by local affiliates of the National Education Association or 
the American Federation of Teachers. School districts ranged in size 
from 101,000 students to fewer than 100. Given this range and because 
I believed school district size may significantly influence the history, 
content, and administration of contracts, I used a stratified random 
sampling procedure consisting of four strata to select 100 school 
districts. Because small town and rural school districts represent over 
62% of all school districts in the state, 1 wanted to make sure that 
adequate samples of suburban, small city, and urban districts' contracts 
were represented in the study. Accordingly, I defined the four strata for 
the selection of contracts based on total student enrollment for the 
district. Group 1 (2501 - 101,000); Group 2 (1001-2500); Group 3 
(501-1000) ; and Group 4 (500 or fewer students). Lfsing a random 
numbers table, 25 districts were selected from each of the four groups. 
The equal “N” per strata does over represent surburban, small city, and 
urban districts in this predominantly rural state. This sampling strategy 
does introduce a possible source of bias. However, if anything, the 
sampling strategy under estimates the generally traditional unionism in 
the state and was viewed as an acceptable trade-off to assure adequate 
samples of contracts in surburban, small city, and urban school 
districts. 

Next I collected copies of the latest negotiated contract for each 
of the identified school districts. It is important to note that at the time 
these contracts were collected and analyzed not all contracts had been 
renegotiated. Because of state imposed revenue caps on local school 
districts limiting salary and fringe benefit for teachers, a number of 
local collective bargaining agreements had remained unsettled. 
However, because the primary issue contributing to various impasses 
between school boards and teacher unions was salary, provisions 
related to teacher professional development generally were not 
affected. Even though a number of districts were operating under 
expired contracts, all contracts examined during this study were the 
existing legal agreements that governed wages, hours, and conditions 
employment for teachers. 
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Structured Interviews. The second phase of data collection 
consisted of 21 structured interviews with superintendents (n=5), 
principals (n=5), teacher union representatives (n=4), directors of 
instruction (n=5), and staff development specialists (n=2). First, I 
identified criteria for the selection of informants. These included 1) 
expertise and experience in teacher professional development; 2) 
leadership position held in the organization; 3) employment in districts 
representing diversity in size, student characteristics, and location 
(rural, suburban, and urban); and 4) employees in districts with 
exemplary professional development practices supported in contract 
language. Key informants were identified using colleague nomination 
and purposive sampling. Using names of individuals identified by 
teacher union representatives, teachers, principals, and other 
administrators, I used the four criteria to select the 21 respondents. 

Based on initial analysis of written contracts, an interview 
protocol was developed to gather more detailed information on the 
influence of specific contract language and provisions teacher 
professional development in local districts, to describe in detail issues 
around contract implementation and professional development, and to 
ask respondents to describe any changes they may have experienced in 
teacher unionism and opportunities for professional growth and 
learning in their schools/districts. 

All interviews were taped and transcribed for analysis. To build 
trustworthiness in the data, written transcripts were returned to 
interviewees for review and editing if needed. Each respondent was 
asked to review the transcripts and to make any changes that he/she 
thought were necessary for purposes of clarity or intent. In general 
their corrections were editorial in nature. Several respondents, upon 
seeing their responses in writing, wrote back that they were somewhat 
surprised, and in a few cases embarrassed, about the lack of clarity in 
their interview responses. However, neither offered clarification in 
their corrected transcripts. 

To enhance credibility, after the first two phases of data 
collection and initial analyses of written contracts and interview data, I 
conducted a foeus group interview with a second set of key informants 
(n=5) that included an assistant superintendent of a large urban district, 
an urban middle school principal, a director of research and 
professional development for a state teachers' association, a special 
education teacher, and a teacher/union representative. Using a 
preliminary set of organizers from these data, the purpose of the group 
interview process was to check initial categorizations of data against 
the experiences and insights of practitioners, to gain a better 
understanding of issues and their implications, and to identify any 
areas not adequately addressed in the examination of union contracts 
and teacher professional development. 

Data Analysis 

Data analysis consisted of two parallel activities. First, a content 
analysis of 100 written collective bargaining agreements was 
completed. The analysis focussed on an examination of such areas as 
specific references to professional development, structures and 
decision making governing teacher development and learning, types of 
professional development activities legitimized in contracts, and 
resources available to support professional development. For the 
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purposes of this study, content analysis focussed exclusively on the 
formal written contracts that governed teacher work and professional 
development. 

To begin the content analysis, I looked for any language 
referring to teacher learning opportunities and professional 
development. This included such terms as in service, staff 
development, training, conferences, and study leaves. This initial 
phase of analysis runs counter to my conceptual definition of 
professional development detailed in the background section of this 
paper. Having said that, this is my conceptualization of professional 
development — not necessarily the one commonly used in schools and 
enumerated in written contacts. Also, teachers, administrators, and 
school boards members often use a variety of terms interchangeably 
when they refer to the concepts of teacher learning and professional 
development. For me, it was important to start with the language that 
currently existed in written collective bargaining agreements and in 
practitioners' ordinary professional discourse. For interview data I used 
a constant comparative method (Strauss & Corbin, 1990) to code data 
and identify themes. Individual and group interview data 
complemented the document analysis by providing details and 
examples of how various contract provisions affected the context of 
teachers' daily work and their professional learning. 

Limitations 

Focusing on written collective bargaining agreements alone has 
limitations. For example, there is more to negotiated agreements 
between teachers and school boards than what is written explicitly in 
contracts. Trust agreements, waivers, joint committee work, and 
district and school policy memuals are examples of other written 
documents that describe and affect these teacher union/school district 
relationships. Collection and analysis of these documents were beyond 
the scope of this investigation. To mitigate the negative aspects of this 
limitation, I believe the collection of interview data was helpful. A 
second limitation concerns the selection of interviewees. Though great 
care was taken to define and select a substantively representative 
sample of teachers, administrations, and union representatives, there is 
always the possibility that the sample does not adequately represent all 
parties to teacher/school district collective bargaining agreements. 
Notably absent from the interview sample are school board members. 
Though important to the negotiation of contracts, the study focussed 
on existing written contracts and their administration which in most 
school districts is left to teachers and administrators in schools. 

Findings 

I use four organizers to present detailed descriptions of the links 
between and implications of union contract language and teacher 
professional development. These are 1 ) contract language and points 
of leverage; 2) the inclusion of professional development in written 
contracts; 3) the governance of teacher professional development; and 
4) changing the professional development paradigm: rethinking, 
restructuring, and reculturing. 

Contract Language and Points of Leverage 
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Despite the rhetoric in educational reform reports that teacher 
professional development is critical to school improvement and reform 
efforts, explicit language on professional development is notably 
absent in approximately three fourths of the contracts examined. Using 
the widest possible net to capture any language and/or activities related 
to teacher professional growth and development, only 28 of the 100 
contracts examined contained any direct reference to teacher 
professional development. Of these 28, only 3 linked professional 
development to district goals and priorities. The finding that teacher 
contracts are generally silent on professional development should not 
be surprising since it is not a mandatory subject of collective 
bargaining. Only recently have examples of localized reform efforts, 
waivers and trust agreements, affecting the contractual relationship 
between teachers and school districts emerged. Yet it is difficult to 
imagine how language covering wages, hours, and conditions of 
employment would be unrelated to teacher professional development. 
In general, even teacher collective bargaining agreements with explicit 
language governing teacher professional development tended to 
remain narrative museums reflecting a legacy of conflict, mutual 
mistrust, legalism, and top-down hierarchies of control over teachers 
rather than expressions of a new unionism. The following are 
examples of explicit contract provisions describing teacher 
professional development primarily under the direct control of 
administrators. 

• The parties agree to establish an In-Service Educational Staff 
Development Committee composed of a representative 
appointed by the Association from each school and no more 
than an equal number of representatives appointed by the 
Superintendent of Schools. The Committee shall assume the 
responsibility for the planning and conducting of the in- service 
and staff development programs for the professional teaching 
staff, subject to the direction and control of the Superintendent. 

• Teachers must fulfill twenty-two and one-half (22 1/2) hours of 
staff development each year. The District may direct up to seven 
and one-half (7 1/2) hours of specific staff development for 
designated teachers or groups of teachers. 

• As required by [... Law], there shall be a regular and continuing 
in-service program which shall be formulated by a standing 
committee composed of administration and faculty members. 

In a few districts the teachers' union and school board have 
negotiated language, in accords or waivers, that recognizes teachers' 
responsibility and control over their own growth and development. 

• The parties to this ACCORD recognize the importance of 
individual growth and development of professional educators 
and the growth and recognition of teaching as a profession. 
Professional educators are responsible for continued 
professional growth through participation in staff development 
activities, formal academic study, and personal enrichment in 
their teaching field and in education in general. 

• Acknowledgment is made of the need for professional growth 
and the consistent need for all teachers to continue their formal 
studies and other related professional activities toward an 
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improved and up-to-date quality instruction. It is also recognized 
that professionalism is an individual decision for teachers and, 
therefore, there is no credit requirement within in any specific 
time period. 

• Teachers are encouraged to continue their professional growth at 
their discretion. 

Collective bargaining for teachers has been a part of state 
statutes since 1959. Reflecting what Kerchner described as second 
generation unionism, the emphasis in the early days of teacher 
collective bargaining centered on increasing salaries, broadening 
benefits, and salary equity. With regard to salary equity, unions sought 
to eliminate capricious, unilateral school board decisions around 
teacher pay and benefits. What's clear in an examination of these 
written contracts is the legacy of these early bargaining days when two 
important principles around teacher compensation and career 
advancement were established. The first was the establishment of a 
legal process for negotiating teacher pay. The second was the 
development of a salary schedule that recognized years of teaching 
experience and advanced educational training as criteria for salary 
increases. The latter is particularly important. Even during years when 
increases in base salary were small, teachers could still increase their 
salaries through professional training and the accumulation of credits 
or the completion of an advanced degree. Thus, linking salary 
increases to advanced training provided an extra incentive supporting 
on-going professional development for teachers. This typifies the type 
of win-win compromise often negotiated by two parties during 
collective bargaining. Advanced training became the solution to satisfy 
teachers' demand for higher salaries and school boards' desire to have 
highly qualified and better trained teachers. 

Leverage Points in Teacher Contracts. 

Notwithstanding the silence surrounding teacher professional 
development in most union contracts, there are a number of negotiated 
provisions, leverage points, that directly affect teacher learning in the 
work place. For example, extra contract days, designated in-service 
days and times on school calendars, hiring new staff, the orientation of 
probationary teachers, teacher evaluation procedures, credits for 
recertification, and extended contracts are leverage points in contracts 
that support teachers' professional development. Interview respondents 
described how various provisions covering teachers' hours and work 
days at times limited what principals and their professional staffs were 
permitted to do contractually, especially as they worked to develop 
standards-based school reform. 

Many of these limitations must be viewed within the context of 
broader political issues at the state level. In particular, the tension 
described by interviewees most often reflected teachers' frustration 
with currently state-imposed caps on teachers' salaries and fringe 
benefits. To control costs in education, in 1994 the legislature 
instituted a revenue cap of 3.8 % of the previous year's budget on all 
local school districts. The only way a school district could exceed the 
revenue cap was to go to a public referendum asking to exceed the cap. 
In effect, teachers' salaries were severely limited. 

Cost controls have also influenced teachers' preferences and 
choices of professional development opportunities as well as its 
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design, delivery and content. With severe limits on their salaries, 
teachers tend to view advancement on the salary schedule through 
rapid accumulation of graduate credits as one of the only ways to 
increase their salaries significantly. Interestingly, this strategy by 
teachers has implications for calculating the total cost of professional 
development. \VTien salary increases due to advanced training are 
included as costs in annual professional development budgets, the 
percentage of the total budget used to support on-going professional 
learning increases dramatically. Additionally, since the revenue caps 
are on the total budget, savings in the budget must come from other 
expenditures. Analyses of written contracts and interview data 
indicated that the resources used to support professional development 
activities typically fund a vast collection of fragmented, individualized 
experiences with little evidence of a systemic focus on district or 
building goals. 

A second leverage point in contracts influencing teacher 
professional development is in the area of teacher leaves. Besides sick 
leaves and those leaves for emergencies, family and extenuating 
circumstances, 72% of the contracts contained provisions for 
professional leaves, study leaves (36%), and personal or sabbatical 
leaves (52%). In most cases, these leaves were unpaid. Less than a 
third of these contracts required teachers to return to the district after 
the leave. Only 13 districts provided any financial support for 
professional leaves. The following language illustrates clearly how this 
provision in the contract supports teacher development. 

Extended Leaves of Absence: Advanced Study 
Purpose: The underlying philosophy of the leave is to 
increase the quality of teaching and to gain enriching and 
broadening experience by professional study and research 
in areas that will promote the employee's teaching ability. 

Major consideration must be given to the benefits which 
will accrue to the pupils and to the community through the 
individual teacher's personal growth. 

In 77 contracts, extra days and extended contracts were another 
important leverage point that supported teacher professional growth 
and development. Administrators, school board members, and teachers 
agree that school success and improvement require on-going training 
and development opportunities for teachers. However, teachers' work 
days provide little time for extensive training or for school 
improvement work. Thus, extra paid days/hours and extended 
contracts for summer work have become critical to meeting the 
training needs of teachers and professional work beyond the 
classroom. Aligning district curriculum to new, state- mandated 
curriculum standards and tests, the introduction of new technologies, 
sundry educational reform initiatives, and more diverse students 
populations, to name a few areas, all require more teacher training. 

Analysis of school calendars attached to these contracts provides 
evidence that districts recognize the importance of time needed for 
teacher professional development. This includes inservice days, early 
release and late starts, and teacher convention days. Ostensibly these 
times and days have been set aside for teacher inservice throughout the 
year. Most districts have 1-2 days per year while a few have schedules 
with weekly early release giving teachers 2 hours for joint work, 
planning, and professional development. At first glance, the number of 
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days and times suggest that districts through negotiations have taken 
seriously the call to provide more time for teacher professional 
development. However, on closer examination it appears that in many 
districts the days set aside for teacher in-service and development 
opportunities have been hijacked. For example, administrators often 
convert these days, especially those scheduled the first day of the year, 
into extended faculty meetings to cover district/school business. These 
days are what Bredeson and Johansson (1999) refer to as “information 
showers” where the focus is on the dissemination of information, not 
teacher learning and growth. Similarly, teachers wanting and needing 
more time pirate inservice days and times to work alone to set up their 
rooms at the beginning of the school year, to complete grades at the 
end of quarters and semesters, and to clean out their rooms for summer 
breaks. 

A fourth important leverage point in written contracts is in the 
area of supervision and evaluation. In 21 contracts, teachers and school 
boards had created alternatives to traditional classroom observations 
and written evaluations of “stand-up” teaching performance. In these 
districts, teachers who completed their probationary years, usually 1-3 
years, could choose a self-designed professional improvement/growth 
plan as an alternative to traditional evaluation. In cooperation with 
principals and supervisors, teachers submitted professional growth 
plans and goals that became the primary basis for their written 
performance evaluation required by law once every three years. 

The Inclusion of Professional Development in Contracts 

Given the small number of contracts that contained explicit 
language on teacher professional development (28%), it was important 
to ask respondents their views on whether or not contracts should 
contain such provisions. Twenty of the 21 interviewees and all of the 
focus group interviewees (5) agreed that teacher professional 
development should be part of union contracts. First, opportunities for 
teacher learning in school and beyond are linked to wages, hours, and 
conditions of employment, all mandatory subjects of collective 
bargaining. To these respondents putting explicit language in contracts 
about professional growth and development in contracts was needed to 
send a powerful substantive and symbolic message to the whole school 
community. 

The general agreement among respondents that language on 
professional development needed to be put in contracts was not seen as 
a silver bullet to improve the design, delivery, and outcomes of teacher 
professional development in schools. As one principal put it, “I mean 
it's probably ok, what we have in here, but...umm.... I think when you 
can establish the right the culture in a given school and school district, 
these things get taken care of ....ah.... far beyond the letter of what's in 
the contract.” Another principal added, that the improvement in 
learning opportunities for teachers in schools needs more than just a 
line or two in the contract. Establishing strong norms and beliefs about 
on-going professional learning in order to improve student learning 
was not something, however, that could be easily specified in contract 
language. “I think it [is a matter of being] pertinent to their [teachers'] 
reality.” “And I think how you just embed that in the everyday work, 
just spills over so naturally, so that these don't even get looked at us 
requirements.” The respondents agreed that teacher professional 
development should be included in negotiated agreements. They also 
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believed that the key to successful staff development for teachers was 
instilling the belief that the time and effort put into the learning 
activities would directly benefit their practice and improve student 
learning. They believed putting language in contracts would help to 
highlight the importance of teacher learning to school improvement 
and student learning. 

The consensus to include professional development in contracts 
was not without some notes of caution. For example, some 
respondents worried that teacher growth and development might fall 
into grievance processes and thereby be rendered ineffective. Others 
worried that by specifying professional development in the contract 
some school boards and teachers' unions might bargain away what 
should be a professional responsibility and attitude among teachers, 
not forced compliance to the strict letter of the contract. One director 
of instruction cautioned, “What you have to be careful of is not to use 
that and end up reducing it [professional development] to, like the 

lowest lowest common denominator.” Another respondent echoed 

the idea that if teacher professional growth and development described 
in contracts became overly prescriptive, such as traditional district in- 
service, there would be much less flexibility and fewer opportunities 
for teacher learning in school and beyond the teaching day. In 
assessing the benefits of putting explicit language on professional 
development in contracts, these educators also acknowledged the 
potential downside if contract language resulted in narrow, 
prescriptive provisions. Such provisions would likely lead to 
minimalism and mere compliance rather than fostering possibilities for 
professional learning; this would be worse than what many 
districts/schools already had. 

Governance of teacher professional development 

Analysis of the contracts and interview data revealed that 
professional development for teachers continues to be top- down and 
primarily controlled and driven by administrators. Only 3 of the 100 
district contracts examined stipulated a full-time coordinator for 
professional development. For mid-size and larger districts, staff 
development fell under the general job duties of curriculum directors 
and assistant superintendents. In smaller districts, the superintendent 
controlled the budget and was seen as the gatekeeper for professional 
development. Regardless of district size, school principals were most 
often viewed as the person primarily responsible for professional 
development. Further examination of contracts indicated that only 17 
districts had formalized in contract language district staff/professional 
development committees composed of teachers and administrators. 
Given the general absence of professional development language in 
contracts this may not be surprising. Yet, even in contracts where extra 
pay for extra duties was described, staff development committee work 
or membership was not included. 

In general, the lack of voice for teachers in decisions around 
their professional growth and development has resulted in a type of 
dependency. As a middle school principal opined, teachers continue to 
think others, administrators in the district, will tell them what to do. 
Thus, there is a fair amount of cynicism about the value of traditional 
professional development in districts. One principal saw this type of 
dependency as professionally debilitating because it has resulted in 
some teachers not even being able to imagine what it might be like to 
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be responsible for planning, implementing, and evaluating their own 
professional growth. Principals believe they have primary 
responsibility for teacher professional development, but in a 
supportive role not a controlling or limiting one. Principals believed 
they had the responsibility and ability to garner the resources, time, 
money, space, expertise, and other resources to support what teachers 
needed to enhance their learning and performance. Principals saw their 
primary responsibility as helping teachers, individually and 
collectively, keep their eyes on the big picture. The principal's role was 
to help align individual and collective teacher needs and interests with 
school priorities and goals. 

Changing the professional development paradigm: Rethinking, 
Restructuring, and Reculturing 

When asked to think about changes they had experienced in the 
past 10 years in the area of teacher professional development, the 
respondents' comments were a mix of optimism and disappointment. 
Each respondent could point to specific examples of positive changes 
in their schools and districts in the area of teacher learning and 
professional growth on the job. However, most were also concerned 
that traditional obstacles remained, many seemingly intractable. To 
improve opportunities for teacher professional development, these 
educators identified three broad areas of change that need to occur 
concurrently. The first is rethinking the current professional 
development paradigm. The second is restructuring the design, 
delivery, content, context and expectations for outcomes of teachers' 
professional development. The third is reculturing schools and 
communities in ways that create and nurture opportunities for on- 
going, job-embedded teacher growth and professional development. 

Rethinking the professional development of teachers. Changing 
the paradigm of teacher professional development requires 
fundamental shifts in the ways teachers, administrators, and 
community members think about its nature, purposes, and goals. To 
begin, respondents agreed that professional development should not be 
seen as an add-on to teachers' work but rather an essential part of what 
teachers do as professionals. Because training and development are 
essential to teachers' professional practice, the resources that support 
them should not be easy targets for budgetary cuts during fiscally tight 
times. The constellation of formal and informal opportunities for 
teachers to leam and to improve their professional craft is crucial to 
school improvement and student success. Thus, in-service, staff 
development training, teacher networks, and collaborative inquiry are 
not just about teachers, they are linked tightly to and aligned with 
school goals and student learning. 

The ways in which teachers and others talk about teacher 
professional development also requires some re-thinking. When in- 
service days or early releases are described as “time-off’ or “wastes of 
time” from teachers' real work, e.g., direct contact with children, such 
expressions communicate the limitations and persistence of the 
traditional professional development paradigm. Teacher professional 
development is legitimate work even when it occurs during the school 
day. Staff development is “time on” not “time off.” 

Another change in thinking is conceptualizing on-going learning 
and development as a professional responsibility. Continuous learning 
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is an essential part of one's professional practice, not just a scheduled 
event or an activity to simply attend and endure. The artificial 
separation of teaching practice and teacher growth and development 
has contributed to the latter. To improve what they do and how they do 
it, reflective teachers rely on their daily teaching experiences to learn 
more about their practice. Traditional school structures and cultures, 
especially self-contained classrooms, have unfortunately tended to 
reinforce teacher isolation and individualism so that the benefits of 
reflective practices remain limited to a few individuals rather than 
becoming part of organizational learning and improvement. In 
addition, the objectification of professional development as something 
“out there” has promoted a type of dependency in teachers often 
leaving them voiceless in planning, implementing, resourcing, and 
evaluating their own learning. “It's completely foreign to them 
[teachers] because they've really had no opportunity to ever have any 
input on anything.” So teachers wait to see, “What's the principal or 
learning coordinator going to tell us to do? What is the district telling 
us we have to do today?” (Middle school principal). Though teachers 
clearly have preferences and know what would be most helpful to 
them in the classroom, traditional designs and delivery of teacher 
professional development in schools often reinforce a dependency 
model in which teachers cede responsibility for their own growth and 
development to others: most often superintendents, principals, and 
staff development persoimel. 

Rethinking teacher professional development also means 
reconsidering the “one size fits all” training and inservice activities 
common in many school districts. In complex school systems there are 
occasions for system-wide informational sessions and inservice 
programs. The key issue is whether or not these types of activities 
dominate staff development activities in schools. Because these 
activities are easier to plan, more economical, and more easily 
controllable, districts frequently default to “one size fits all” sessions. 
Undifferentiated training sessions rarely provide learning opportunities 
that engage individual teachers' creative and reflective capacities to 
strengthen their practice. 

Restructuring the professional development of teachers. “We 
haven't admitted that we're going to have to blow the thing up in order 
to get real fundamental professional development in the system.” 
Either metaphorically or concretely, changes in thinking about teacher 
professional development need to be accompanied by fundamental 
changes in the structures that support it, including provisions in 
contracts. For example, respondents described how schools and their 
operations need to be reconfigured to permit more time within the 
school day for teacher learning. As one respondent said, “A couple of 
things are happening that I think damage our opportunities to really 
change things in a significant way. One is the issue of the structure of 
the school day, the school week, and the school year. There is no 
collaborative time structure in our work. And, teachers are too isolated 
in their work. That we have to really redesign the system in a way that 
guarantees that the time is there for collaboration; and 1 don't think this 
society is willing to pay for that, and that's been our big 
problem” (Professional Association Representative). The issue of 
restructuring time, with a focus on the use of time and its impact on 
conditions of employment, has important implications for collective 
bargaining and opportunities for teacher learning within contract days. 
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Time was described by respondents as the most important structural 
factor that needed to be addressed in order to change the current 
professional development paradigm. In a few written contracts there 
are appended waivers and side- agreement that supported changes in 
the use and structure of time in the teacher's work day. The interview 
data provided ample evidence that a number of schools and districts 
were using collaborative informal agreements, between administrators 
and teachers, to address teacher learning in newly configured time- 
frames. This includes such practices as 1) providing teachers extended 
contract days over the summer; 2) extra pay for committee work that is 
beyond ordinary teacher work expectations; 3) hiring substitute 
teachers, both permanent and temporary, so that teachers have time 
during their the school day to meet and work together; 4) early releases 
and late starts for students; 5) scheduled staff development days; 6) 
creative use of class time through block and flexible scheduling; and 
7) banking time, e.g. increasing class periods and school days several 
minutes a day to bank time for future release times. 

According to these educators, even when time is available there 
may not be a place in the building for teachers to meet and work. 
Outdated buildings, the proliferation of programs and specialities to 
support students beyond the classroom, and overcrowded schools often 
leave teachers in hallways or other cramped spaces, hardly optimal 
conditions for professional learning. Clearly this is less of a problem 
when students are released, but if professional development is to be 
embedded in teacher work, creating learning spaces for teachers is an 
important part of restructuring schools. Providing physical space for 
teacher learning in schools also sends a powerful symbolic message 
about the importance of continuous growth and development in 
schools. Conferences rooms, office space, work rooms, labs, and basic 
communication tools (computers, telephones, and fax machines) are 
minimal requirements for any professional, yet these tools are scarce 
in most schools. Additionally, the lack of these basic resources to 
support teacher engagement, reflection, and growth reinforces norms 
of privacy, isolation, and dependency that threaten the development of 
an authentic professional learning community. 

The reallocation of resources to support new conceptualizations 
and practices in teacher professional development is also an important 
element in restructuring. In some districts, teachers and administrators 
have creatively knitted together a mix of local, state, federal, and 
private monies to support professional development and school change 
processes. In others, however, the patchwork of traditional 
development programs and activities, and the budget lines to support 
them, are not clearly aligned with district/school funding priorities. 
Because of revenues caps in the state, most districts are actively 
seeking external grants to support staff development for teachers. In 
some districts, it is unimportant what the focus or goal of the funding 
agency is. They become “Christmas tree districts” where teachers, 
administrators, and school boards willingly subordinate local priorities 
and goals for high profile programs that send extra dollars to support 
teacher development and training opportunities. When the typical 
three-year funding cycle ends, so does the initiative. The district then 
reinvents itself in order to respond to new criteria described in another 
request for proposal (RFP). 

New state mandates, especially the newly adopted model 
academic standards linked to legislated testing of all students, have 
intensified change efforts and completely dominated staff development 
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and in-service training across the state. A union representative from 
one suburban district noted, compliance with these state mandates, 
especially activities focussed on curriculum alignment and organizing 
for testing in four core areas, is robbing teachers and their schools and 
districts of what little time, energy, and resources that had already 
been set aside for professional development. 

Selecting and hiring teachers with a professional orientation 
toward their own growth and development was cited by principals and 
superintendents as an important structural piece that supported teacher 
professional development. The administrators believed one of their 
primary responsibilities was to establish criteria and develop processes 
that enabled their districts/schools to identify and hire candidates who 
viewed continuous growth and development as an essential part of 
their professional work and one for which teachers took responsibility. 

Restructuring the delivery of teacher professional development 
is also critical to changing the current paradigm. Long dominated by 
workshops and fragmented in- service meetings, new forms of 
professional development have emerged with “a much deeper and 
more sophisticated focus on instruction.” The idea of “one size fits all” 
is fading away. In districts with leading edge practices, teacher 
professional development tends to be more localized, more centered on 
individual teacher needs, carried out in interactive and participative 
settings, and is on-going and long-range in focus as opposed to one- 
shot presentations and events. Such practices are beginning to break 
down teacher isolation and build learning communities among 
professionals seeking to improve their practice, not simply acquiring a 
few “nuggets of knowledge” for easy transfer to classroom teaching. 
Teachers and principals are attempting to redesign the school “Culture, 
climate.. .ah. ..the way we structure our interactions, so that it's 
supporting people's learning everyday.” 

Reculturing teacher professional development. “It's not in the 
culture of our district to have the union talking about professional 
growth and development. The way of talking about it is through 
compensation” ( Elementary principal). Rethinking and restructuring 
teacher professional development are part of the larger process of 
reculturing schools and communities to support teacher learning. As 
the preceding quote indicates, the values, beliefs, and practices that 
define the current culture of teacher professional development may be 
anything but professional. For example, teacher isolation, work days 
with little or no time for professional development, administrator 
dominated planning and decision making, and fragmented staff 
development and training activities typify the current culture. “The 
system needs to be redesigned for teachers to really become active 
learners. Our structure does not facilitate that.” (Teacher union 
representative). Typically, teacher work has been defined as standing 
up in front of and working directly with children. Working directly 
with children is difficult to argue against since teaching children is the 
primary mission of schools. The reculturing efforts described by these 
respondents are meant to enhance teachers' work with students by 
recognizing and incorporating teacher professional development “ as 
professional work” and “at work”. In recultured professional learning 
communities, staff in-service and training days are not days off, they're 
days on. The ways in which teachers and others talk about teacher staff 
development are expressions of reculturing that communicate 
important values, norms, and practices that characterize high quality, 
professionally oriented schools focussed on student success. 
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The ways in which school boards and teacher unions address 
teacher professional development in collective bargaining also help to 
define the culture. With nearly three-fourths of the contracts silent on 
teacher professional development, developing new professional 
learning cultures will not come easily. The baggage of traditional 
unionism, collective bargaining experiences, and grievance arbitration 
in schools has left both parties, teachers and school board, nervous 
about asking the other to dance. As one union representative put it, 
“Nobody knows how to behave.” Learning how to “behave” requires 
trust between parties to negotiate agreements. In districts where new 
professional development cultures have emerged, values, norms, and 
practices are simply embedded, “In the everyday work, [and] just spills 
over naturally, so that these don't even get looked at as requirements. I 
think when you can establish the right culture in a given school and 
school district, these things get taken care of .ah., far beyond the letter 
of what's in the contract” (Elementary Principal). Establishing norms 
of trust requires time and experiences that build on joint commitment 
and efforts among teachers, their unions, administrators, school board 
members, and the community. To date, in only a small number of 
districts have teachers and school boards redefined and renormed their 
formal contractual relationship. 

Conclusion 

There are numerous challenges confronting stakeholders in 
public education at the end of twentieth century. Among these is 
whether teachers and local school boards will be collaborators or 
combatants as they confront a seemingly endless array of problems. 
Perhaps one way to build a bridge to “new unionism” and leave behind 
the baggage of adversarial collective bargaining is through the 
development of professional learning communities in schools. 
Successfully negotiating the uncharted terrain of these learning 
communities for students and teachers requires good will, trust, and 
the commitment of teachers, administrators, and school boards to work 
together. 

Leadership is critically important to the direction and designs for 
new unionism and the growth of professional learning communities in 
schools. Though not exclusively, much of the move toward new 
imionism will come through formal contract bargaining, as well as 
policy initiatives, side agreements, and negotiation of daily work in 
schools. The risks involved, the creativity required, and the mechanics 
of living these newly forged relationships among school boards, 
administrators, teachers, and the people they serve require leadership 
at all levels — ^policy making, contract bargaining, administration, and 
teaching and learning in classrooms. Issues surrounding teacher 
professional development, the focus of this study, will be important 
ones as teachers, school boards, and other key educational 
stakeholders renegotiate formal and informal relationships in schools. 

As the findings in this study indicate, shifts in thinking, 
structures, and organizational cultures are the initial expressions of 
new unionism and the development of teacher professional learning 
communities. These data indicate that most school districts in this state 
are still at the proverbial “starting gate” of new unionism. Local 
experiments are begiiming to emerge. However, the transformation to 
what Koppich and Kerchner (1999) describe as new uriionism centered 
around such organizers as quality, localism, and flexibility in the 
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teacher labor market remain distant. The movement toward a new 
generation of unionism and professional relationships in schools, 
though slow, continues to advance. The findings from this study on 
union contracts and teacher professional development suggest a 
number of areas in which teacher unions and school boards can initiate 
this collaborative venture, though the legacy of second generation 
unionism is deeply rooted in school districts across the state. 

First, the language used to describe teachers' professional 
development and their work is important. Based on the beliefs of 
respondents in this study, highlighting the importance of professional 
learning in negotiated agreements has both symbolic and substantive 
power. However, contract provisions and explicit language are not 
substitutes for actions and practices embedded in the daily work of 
teachers, principals, and others that nurture and support authentic 
professional learning communities. Second, the current professional 
development paradigm is not anyone's the fault. It's the result of a 
shared history. There are a few examples of joint committees, trust 
agreements, and waivers, developing at what Koppich and Kerchner 
(1999) call the margins of union transformation. However, I believe 
these early experiences within schools and districts in the area of 
teacher professional development provide opportunities for trust and 
confidence to develop among educational stakeholders and parties to 
collective bargaining agreements. It is on these experiences that 
dramatic changes in unionism and teacher professional development in 
schools will occur. 

Creating professional learning communities that support and 
encourage teacher professional growth and development over a career 
will require fundamental shifts in the current paradigm of teacher 
professional development. Concurrently three streams of change, that 
resonate with the principles of the most effective professional 
development practices need to be negotiated between teachers and 
local school districts. First, changing the professional development 
paradigm requires rethinking and revisioning the design, delivery, 
content, and outcomes of teacher professional development. 

Rethinking teacher professional development requires the 
collaboration and voice of teachers, school board members, 
administrators, and community members. Rethinking teacher 
professional development and reframing it in teacher contracts is not 
just an issue between teachers unions and school boards. It is a public 
issue requiring the input and understanding of all educational 
stakeholders. A second stream of change is restructuring teacher 
professional development. This restructuring requires a new 
architecture expressed in collaborative, negotiated agreements that 
creatively reconfigure time, space, resources, and materials to provide 
learning spaces for teachers in their work and beyond. Finally, a third 
stream of change is reculturing schools and teacher professional 
development. Reculturing begins with valuing teacher learning and 
understanding its link to high quality schools and student achievement. 
Teacher unions and school boards though their collaborative efforts, 
not confrontational relations, can help students, parents, and other 
community members understand the importance of teacher growth and 
development and its link to school/district goals. 
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Abstract 

The purpose of this study is to examine the historical 
factors which affected the rise of modern higher education 
during the late Choson period (1880-1910), and to analyze 
the implications of these historical factors on educational 
policies in contemporary higher education in Korea. The 
rise of modem higher education in Korea can be viewed 
as occurring in three principal phases: Confucian Choson 
Royal Government, Western Christian missionaries, and 
patriotic nationalists. The author points out that the major 
historical factors influencing the development of modem 
higher education were Confocianism, Christianity, and 
Korean nationalism. In particular, Confucianism and 
Christianity have had substantial impacts on the planning 
of educational policies in contemporary Korean higher 
education; the former is viewed as an original source of 
educational enthusiasm which has expanded Korean 
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higher education, and the latter a matrix of modem 
Korean higher education which has embodied educational 
enthusiasm. 



Introduction 

The Korean people have long respected Confiician learning 
(Note 1) and have attached great significance to education throughout 
Korean history. This tradition began in the three early kingdoms 
period (57 BC-AD 668) and continues to the present time. According 
to one important historical record, Samguk-sagi (Historical Record of 
the Three Kingdoms) (Kim, 1 145), the intellectual activity during the 
Three Kingdoms period was the learning of Chinese thought and 
culture, which was much more highly developed than native Korean 
thought and culture at that time. After that period, the succeeding of 
the Unified Silla Kingdom (AD 668-935) and the Koryo Kingdom 
(AD 918-1392) still maintained Confiician study as a major academic 
field in spite of Buddhist monarchy. In the Choson period 0392- 
1910), the mlers of the Choson Kingdom accepted Confucianism as 
the source of basic principles for national politics, ethics, and 
education for over 500 years. 

From the Three Kingdoms period to the late Choson era, although 
Korean elite or higher education (Note 2) had generally followed in 
the steps of the Chinese educational system, it is clear that the 
traditional or pre-modern higher education systems of Korea and 
China were not identical. As the Samguk-sagi (Kim, 1 145) points out, 
Confiician studies and traditions from China were independently 
integrated into Korean culture and society. However, these traditions 
were modified during the late Choson period. In terms of the history of 
Korean education, the late 19th century was a pivotal period whereby 
the traditional educational system and Confiician elite institutes were 
devalued by Japanese politicians and Western Christian missionaries. 

In light of the periodic importance of Korean education, several 
researchers have studied Korean education during the late Choson 
Kingdom and the Imperial Japanese Administration periods. For 
example, Horace H. Underwood (1926) in his book. Modern 
Education in Korea partially discussed Korean higher education under 
the late Choson Kingdom and Japanese colonial rule. James E. Fisher's 
(1928) Ph.D. dissertation, “Democracy and Mission Education in 
Korea,” explored missionary education in Korea under Japanese 
colonial rule from the viewpoint of Christianity. Han- Young Rim 
(1952) also researched the development of Korean higher education 
under Japanese colonial rule in his unpublished Ed.D. dissertation, 
“Development of Higher Education in Korea during the Japanese 
Occupation (1910- 1945).” Sung-hwa Lee (1958) briefly examined the 
social and political factors affecting Korean education from 1885 
through 1950, and In-soo Son (1985) analyzed Korean education from 
1876 to 1910. 

In this study, I will examine the historical factors affecting the 
rise of modern higher education during the late Choson period (1880- 
1910) and analyze the implications of historical factors on educational 
policies in contemporary Korean higher education. Furthermore, this 
study will provide both Eastern and Western educators with valuable 
information about the development of modem Korean higher 



Page 2 of 15 



httn ://et)aa. asu. edu/ enaa/v9n2 7 .html 



ort 



10/29/01 




EPAA Vol. 9 No. 27 Lee: The Establishment of Modern Universities in Korea 



Page 3 of 1 5 



education from a perspective of educational policy. I will first briefly 
illustrate the historical background of the study from the Three 
Kingdoms period to the late Choson period. Second, I will examine the 
circumstances of modern education during the late Choson Kingdom 
(1880-1910), classified into three phases: royal government operated- 
schools based on Confucianism, missionary schools on the basis of 
Christianity, and native private schools on the grounds of nationalism, 
and then review the historical factors affecting the development of 
modern higher education in Korea. Finally, how the historic factors 
impact educational policies in contemporary Korean higher education 
will be analyzed. 

Historical Background 

It is generally taken for granted that the Korean people attach 
great importance to education. This was true for many centuries and 
continues to the present time. According to Samguk-sagi (Kim, 1 145), 
the first formal institution of elite or higher education was known as 
Taehak (National Confucian Academy), built by the Kingdom of 
Koguryo (Note 3) in AD 372. A similar institution for elite or higher 
education named Kukhak (The National Academy) was established in 
the Silla Kingdom (57 BC-AD 935). The Packche Kingdom (18 BC- 
AD 660) also stressed elite or higher education and produced 
numerous scholars in various academic disciplines, many of whom 
made important contributions to the flourishing ancient Japanese 
culture (Nihongi, Vol. I, Trans., Aston, 1896, pp. 262-63; Kojiki, 
Trans., Chamberlain, 1973, p. 306). Elite or higher education in the 
Three Kingdoms tended to focus on the study of the Chinese classics 
of Confucian orientation in order to establish their aristocratic political 
and social systems. 

Like Taehak and Kukhak in the Three Kingdoms, Koryo (918- 
1392) had educational institutions that educated the elite who led its 
aristocratic society in order to maintain their hereditary political and 
economic privileges. Koryo already had elite schools in the capital, 
Kaeseong, and Pyoungyang in the first King Taejo's reign (918-943) 
(B. Lee, 1986, p. 47; K. Lee, 1984, p. 119). In the 10th year of King 
Seongjong (AD 992), Kukchagam (the National Academy or 
University) was established in the capital. This institution included 
three colleges: Kukchahak (Higher Chinese Classical College), Taehak 
(High Chinese Classical College), and Samunhak (Four Portals 
College). Subsequently, under the reign of King Injong (1 122-1146), 
three colleges were included: Yurhak (Law College), Seohak 
(Calligraphy College), and Sanhak (Accounting College). The six 
colleges all existed under the Kukchagam. 

These colleges had different entrance qualifications, curricula, 
and instructors. For instance, the Kukchahak admitted the sons and 
grandsons of officials above the third rank. The Taehak was open to 
the sons and grandsons of officials above the fifth rank and the great 
grandsons of officials of the third rank. The Samunhak was devoted to 
the sons of officials of the eighth and ninth, as well as the common 
people, who were admitted to study at one of three special colleges: 
Yurhak, Seohak, and Sanhak. The curricula of Kukchahak, Taehak, 
and Samunhak mainly taught the Chinese classics. The other schools' 
curricula dealt with technical subjects — for example, law, Chinese 
calligraphy, or accounting. The instructors of the first three institutions 
were ''Paha (Learned Doctors)” and "Chokyo (Assistant Doctors),” 
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while the instructors of the latter three schools were “Paksa (Learned 
Doctors).” Institutions of higher education were open to the offspring 
of aristocratic families so that they might maintain their political, 
economic, and social privileges. Particularly, the Chinese classics that 
were based on Confucianism contributed much to Koryo society and 
politics through education. 

From the beginning of the Choson Kingdom, Confucianism was a 
national religion. Choson rulers stressed Confucian education to train 
the civilian bureaucrats to lead their people, and to enable the people 
to follow Confucian ethics and values. Seongkyunkwan (Hall of 
Harmony or the National Confucian Academy) was established in the 
capital city during the reign of King Taejo (AD 1392-1398). The 
Seongkyunkwan eventually succeeded all other organizations, 
curricula, and functions of Kukchagam or Kukhak in the Koryo period. 

The students of the Seongkyunkwan, who were the offspring of 
the bureaucrats, consisted of two hundred seng-won (classical 
licentiates) and chin-sa (literary licentiates). The curricula of the 
institution included Ku-che (Nine Subjects), that is, Saseo (the Four 
Confucian Books) and 0-Kyung (the Five Chinese Classics). The 
subjects were instructed by various teaching methods; reading, 
composition, argument, persuasion, praising, and epigrammatic poetry 
(Choo, 1961, p.36). Completing all course work, the yusaeng 
(graduates) were permitted to take the Kwa-keo (the government 
examinations), particularly Dae-kwa or Mun-kwa (Trieimial Higher 
Examinations or Emdite Examinations). 

As mentioned above, in the early Choson Dynasty, elite education 
was chiefly regarded as an institution for preparation of the future 
civilian bureaucrats who then rose to political positions after passing 
examinations {Kwa-keo). Accordingly, since the Seongkyunkwan, as 
the highest educational institution, did not fulfill its function of 
pursuing knowledge and truth, it was reduced to a tool for preparing 
students for the national civilian examination systems. The 
examinations based on the Chinese classics constituted core curricula, 
and teaching methods were predominantly rote memorization and 
writing. Furthermore, even provincial and private schools stressed the 
preparation of the students from the lower civilian examinations or 
advanced studies to take the higher civilian examinations. Choson 
society was mainly directed by the Yangban (high level) class, who 
monopolized politics and the economy of the country; education was 
no exception. Therefore, education was regarded as the ultimate means 
to maintain the Yangban's socio-political privilege, and the Yangban’s 
educational enthusiasm contributed much to the pursuit of the ruling 
class's interest and power. 

Indeed, the Choson rulers used the examination systems to 
protect their own interests. Although the examinations were 
supposedly open to the common people, they rarely passed the 
examinations because the Confucian academies were strictly forbidden 
to the commoners. In particular, women and Sangnom or Cheonmin 
(the mean people) (Note 4) were excluded from the learning 
opportunities in public institutions. Moreover, occupational or 
technical education (Note 5) was ignored by the Yangban class. 
Buddhism, Taoism, and the traditional folk beliefs were not discussed 
in the Confucian institutions and in the Confucian bureaucratic society 
of the Yangban. 

Consequently, the Confucian educational system, which 
depended on the Kwa-Keo as a backbone of the early Choson's 
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education, was maintained until the late 19th century when the Choson 
Dynasty opened its doors to coercive foreign power and received the 
Western modem educational system. Under these powerful influences, 
the Seongkyunkwan, as the highest educational institution, inevitably 
terminated the Confucian educational tradition and, unfortunately, bid 
farewell to the Choson's elite, particularly the Confocian literati. 

Historical Factors Affecting the Advancement of Modern 
Higher Education in Korea 

A new movement which called Silhak (Practical Learning) for 
modernization blossomed during the late 17th to the 18th century. A 
group of Choson scholars sought to devise practical ways to use 
academic knowledge to modernize the state. With the introduction of 
Roman Catholicism and Western knowledge by the Ching Dynasty 
(1644-191 1/1912) of China, Choson scholars endeavored to create a 
modernized country. Unfortunately, these pioneers never reached their 
goal of reforming the Confucian Choson Kingdom politically, 
economically, socially, or educationally because the highly centralized 
bureaucratic politicians ignored the new religion and knowledge. 
Owing to the failure of the Silhak movement, the Korean people lost 
the chance to reform the old educational systems autonomously. 
Therefore, the beginning modern education in Korea was delayed until 
the late 19th century. 

Confucian State Operated-Schools 

In the late 19th century, the Confucian Choson Government 
recognized the importance of Western knowledge and education 
through external coercive power and an internal national awakening. 
Accordingly, in 1883, the Choson Government established 
Dongmunhak (The English Language Institute) as the first 
governmental modem school in the capital. Three years later, the 
government also set up Yukyoung-kongwon (The Royal English 
School) to educate the sons of the aristocratic Yangbans in English and 
other Western knowledge. Although both offered Western education to 
train future interpreters or governmental officials, they strictly kept the 
traditional Confucian educational systems and curricula. The two 
schools were not actually the types of modern educational institution 
needed to meet the demand for a new education at that time. 

However, before the Confucian Choson Government could 
reform traditional education by itself, political and social reform 
(Kabo-Kyungjang) was carried out in 1 894 as Japanese political forces 
demanded reform of the political, economic, and social systems of the 
Choson Kingdom. It is a widely held belief that the Japanese planned 
the occupation of the Korean peninsula as an advanced base to invade 
the Asian continent. Thus, the weak Choson Royal Government had to 
carry out huge reform— political, economic, social, and educational- 
under irresistible Japanese political pressure. The reforms in the social 
and educational systems included the abolition of the social status 
system, discontinuance of the Kwa-Keo (the government 
examinations) system (Brown, 1919, p. 79), and the creation of new 
educational systems from primary and secondary to vocational and 
foreign language schools. In particular, the Royal Government 
recognized the significance of teacher training as a means of 
modernizing education on the foundation of primary education. In 
1895, according to the Royal Prescript of Education, Hanseong 
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Teacher's School was established in Hanseong (present-day Seoul). At 
the same time, old educational institutes, except Seongkyunkwan (The 
National Confucian Academy), were officially abolished. 

On the other hand, although the occupational schools such as 
medical, law, commercial, foreign language, and technical institutes, 
were established, these schools were not highly regarded by the 
Korean people, particularly the Yangban (the ruling class) who 
despised occupational and technical skills. After the Kabo Reform 
(1894 Reform), the Choson Government tried unsuccessfully to 
change the old educational systems into modem Western types. 

Despite such resistance, some Christian missionary schools and native 
private schools were the seeds from which contemporary Korean 
higher education grew. 

During the late Choson period, the Royal Government did not 
have sufficient finances to establish the highest educational institution 
as a Western modem university, nor was it familiar with Western 
higher educational systems (Bishop, 1 897). Institutions which were 
founded by the Choson Government were typically elementary and 
secondary level schools. Therefore, there were no state operated 
schools resembling the modem Western university during the late 
Choson times. 

Western Christian Missionary Schools 

The second type of school founded by Western Christian 
missionaries greatly contributed to the development of modem 
education in Korea. In particular, the Catholic missionaries were 
educational pioneers who taught the native letters, namely han-geul, to 
the Korean women and men of humble birth for the understanding of 
Christianity before the Protestant missionaries arrived in Korea in the 
late 19th century. Along with the first Korea-U.S. Treaty on May 22, 
1882 (Allen, 1908), a number of Christian missionaries of different 
denominations, Protestantism in particular, arrived and started medical 
as well as educational institutions as ways of carrying out their 
missionary work (Underwood, 1926, p.l3). 

The first American Presbyterian mission was opened by Dr. and 
Mrs. H. N. Allen who arrived in Seoul in September 1884 (Allen, 

1908; Mckenzie, 1920). In the spring of the following year, the Rev. 
Horace G. Underwood, who published the first Korean-English and 
English-Korean dictionary, landed. The Rev. and Mrs. Henry G. 
Appenzeller, as well as Dr. and Mrs. Scranton with Dr. Scranton's 
mother, Mrs. Mary F. Scranton, of the Foreign Missionary Society of 
the Methodist Episcopal Church also arrived (Gale, 1909, pp. 161-63; 
Underwood, 1926, p. 9). Other American and Western missionaries, 
including Australian, English, and Canadian, also arrived in Korea 
before the end of the 19th century (Mckenzie, 1920, p.205). 

In 1885, Dr. Allen established the first Western modern hospital 
called Kwanghyewon (the National Hospital) as a Mecca of medical 
education in Korea to provide education along with clinical practice. 
The hospital was to become a cornerstone for the Severance Union 
Medical College (the predecessor of the present Yonsei University 
Medical College), which opened in 1903 (Underwood, 1926, p. 120). 
(Note 6) In 1886, Mrs. M. F. Scranton opened the Methodist Girls' 
School (Ehwa-hakdang) as the first girls' school in Korea, which 
evolved into the present Ehwa Woman's University. (Note 7) Although 
the school began with one student, it gradually began to play a 
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significant role in emancipating Korean women from the rigidly male- 
dominated Confiician Choson society, giving the females valuable 
opportunities to learn through both traditional and modem education. 
After Mrs. Scranton founded her school for girls, Rev. Appenzeller 
established Baejae-hakdang, the first missionary high common school 
for boys in the country, on June 8, 1886 (Bishop, 1897. p. 388; 
Underwood, 1926, p. 18).(Note 8) In 1897, Sungsil School was 
founded by the U.S. Presbyterians (Northern) at Pyungyang; and 
subsequently the school, called Sungsil Union Christian College (now 
Sungsil University) in 1906, was first developed as an international 
and union college in which the Northern Presbyterians, Northern 
Methodists, and Australian Presbyterians cooperated (Underwood, 
1926, p. 127). 

After Sungsil Union Christian College had operated for several 
years, many Christian missionary schools (Note 9) were established 
under different denominations and missions. All Christian schools or 
colleges stressed the evangelical ministry, although humanistic £md 
natural sciences were also taught. In the early stages, most students of 
Protestant missionary schools came from the non-Yangban class, 
including women and the lower classes. Nonetheless, there is no doubt 
that these Christian missionaries sowed the seeds of Christianity and 
democratic spirit to the Korean people through education. The 
Christian missionary work in Korea had a great effect on the 
development of Korean education, including higher education in the 
following ways: awakening the national spirit, disseminating 
Christianity, recognizing the importance of Western practical and 
scientific loiowledge, opening democratic education and education for 
women, teaching Western institutional systems and curricula, and 
instituting native language education. 

Native Private Schools 

The third type of institution was established so as to encourage 
national spirit and to enhance national power by farsighted patriots 
who intended to protect their country and people against foreign 
imperialists. The founders recognized new ideas which would help 
their nation to become modernized and stressed the role of education 
in developing a powerful country, politically and economically. 

The first modern private school called the Wonsan-Haksa 
(Wonsan Academy) was established by the magistrate of Teogwon 
county and local residents at Wonsan in 1883 to serve the growing 
interest in education of the young (K. Lee, 1984, p.330). The school 
partially taught the traditional Chinese classics at first and then 
gradually included foreign language, law, geography, and international 
law. Twelve years later in 1895, Younghwan Min opened Heunghwa 
School, which primarily taught English, Japanese, and land surveying. 
Between the 1 890s and 1 900s, the native private schools mushroomed 
in the capital and provincial areas. However, the schools generally 
belonged to the secondary school or primary college level. Thus, there 
was no higher educational institution like a Western modem university 
in operation. Indeed, among the above private schools, only Boseong 
School, which was founded in 1905, became Boseong Junior College 
(as a predecessor of the present Korea University) during the Japanese 
colonial period. 

In the late Choson period, although many patriotic leaders 
promoted the establishment of higher educational institutes to reform 
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their under-modernized country, they could not overcome Japanese 
political power. Especially, after the 1905 Protectorate Treaty was 
signed, the Choson Govenunent practically lost its national right to 
govern. During the “Protectorate” period (1905-1910), the Japanese 
educational policy was chiefly the preparatory operation for 
colonization through the promulgation and practice of various 
educational ordinances and regulations. For instance, the Private 
School Ordinance that was promulgated in 1 908 was a means of 
placing under Japanese control and suppression all those private 
schools administered by Christian missionaries and patriotic Korean 
leaders (Underwood, 1926). 

Therefore, the native private institutes under the Japanese 
“Protectorate” period lost opportunities to plant Western models which 
were suited to Koreans' needs due to the Japanese imperialists' 
educational policy to accomplish the annexation of Korea. 

Furthermore, Japanese imperialists attempted to abolish the native 
private schools as well as the Confucian institutes that had preserved 
the Korean academic tradition. They regarded education as a tool of 
assimilation to Japanese culture and of dampening of the Korean 
national spirit. 

In terms of the development of Korean higher education, 
although the native private schools did not offer modem curricula and 
organizational stmctures of higher educational institutions because of 
Japanese political power, there is no doubt that the schools, as the 
preliminary institutions of higher education, marked a clear turning 
point in the history of modem Korean education and laid a cornerstone 
for the native private institutes in Korean higher education and 
encouraged the Korean national spirit. 

In sum, the Choson Government recognized the significance of 
Western practical and scientific knowledge to the creation of a 
powerful country and carried out political, economic, social, and 
educational reforms; but the Government did not achieve these as a 
result of the prejudice of the conservative Confucian literati about 
Christianity and Korean people's general ignorance of Western 
knowledge. At that time, neither did most Korean people have an 
interest in the governmental education, nor did they willingly follow 
the governmental reforms controlled by the Japanese. Indeed, many 
conservative Yangbans wanted to maintain the Confucian educational 
tradition rather than to accept Western education instigated by the 
Japanese. After 1900, many young Korean nationalists were interested 
in the institutes founded by the national patriots, who had been active 
in political and educational endeavors, while the Confucians kept their 
conservative traditions. However, traditional Confucian schools 
gradually decreased in number due to the increase of Christian 
missionary schools and the native private schools. 

The Implications of Historical Factors on Education 
Policies in Contemporary Korean Higher Education 

Historical factors such as Confucianism, Christianity, and 
nationalism had a great influence on the development modem higher 
education in Korea. In particular, religion has played a significant role 
in the plarming of the national education policy as well as in the 
development of contemporary Korean higher education. With 
Buddhism, Christianity has become one of the two representative 
religions (Note 10) in contemporary Korean society and leads the 
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private colleges and universities in current Korean higher education. 
Confucianism did not contribute to establishment of Confucian 
institutes in modem Korean higher education; but it maintains a 
constructive relationship with Buddhism and Christianity. As 
Confucianism and Buddhism coexisted in Korean society as primary 
or secondary institutions until the late nineteenth century, so 
Christianity has mainly followed Confucian socio-ethical ideologies as 
a new adopted cultural mediator since the late nineteenth century. In 
contemporary Korean higher education, Confucianism and Christianity 
are two main historical factors dominating organizational culture as 
well as educational administration. In addition, the two factors are the 
main pillars of the planning of national education policies internally 
and externally. Confucianism has contributed to the planning of 
organizational structure and culture, whereas Christianity has 
contributed to the planning of instructional curricula and 
administrative systems. 

In the history of Korean culture and education, Confucianism 
was a primary or secondary key institution in formal Korean elite 
education until the late nineteenth century. Christianity, on the other 
hand, had a significant impact on the introduction of modern higher 
education in Korea through harmonizing the religious and educational 
traditions of Confucianism during the late Choson period. Christianity 
still plays an important role in private postsecondary institutions. 
Confucianism does not directly contribute to the development of 
current Korean higher education, but Confucian socio-ethical 
principles and values are the principal axes of organizational culture in 
higher education administration. 

In contemporary Korean society, the two ideologies — 
Confucianism and Christianity — coexist under the aegis of 
democratization and industrialization. With educational zeal based on 
the adoration of Confucian learning, Christianity and Western ideas, 
especially democratic and scientific approaches, brought about 
significant economic and educational advances. However, despite this 
positive side, a negative side also exists. In particular, mammonism 
and egoistic individualism threaten traditional values and norms. With 
the rapid expansion of Korean higher education, however, the 
influence of valuable traditional thought has gradually diminished, 
while individualism, materialism, utilitarianism, and scientism based 
on Christianity and Western ideas have spread broadly through Korean 
society. In addition, higher education is reduced to the level of being 
simply a tool to accomplish the individual's socio-economic 
aspirations while ignoring human ethics and morality. As a result of 
these phenomena, the traditional humanitarian spirit based on 
Confucianism is threatened by educational acculturation and pragmatic 
scientism. In addition, the educational zeal that has been a significant 
social factor or a driving force in the expansion of contemporary 
Korean higher education has been reduced to a means for individual's 
success. 

Throughout the history of Korea, educational enthusiasm 
originated in traditional Confucian education. Of course, this 
enthusiasm is also found in other East Asian countries which follow 
the tradition of Confucianism. However, Korea's unique historical and 
cultural background resulted in the Korean people's adherence to 
Confucianism as the state religion for over 500 hundred years until the 
early twentieth century. 

Educational zeal in past eras was viewed as a desire to maintain 
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the Yangbans' socio-political privileges. This educational enthusiasm 
contributed to the pursuit of the Yangbans' socio-political interest and 
power through Confiician elite education and the state examination 
system, while maximizing the instrumental values of education. In the 
later nineteenth century, however, Confucian elite education declined 
in traditional functions and began to convert the Yangbans' monopoly 
to the commoners' concerns. 

During the Japanese colonial period (1910-1945), owing to the 
Japanese colonial education policy of "Japanization," it was only 
possible for Japanese and a small minority of Koreans to access higher 
education. Most Koreans able to participate in higher education were 
pro-Japanese Koreans or the former Yangbans. Although a few 
common people could access higher education after to the abolition of 
the strict social hierarchy, most Koreans did not readily abandon 
traditional Confucian values and education. For this reason, it was still 
difficult for the Korean populace to participate in higher education 
during Japanese colonial times. 

After liberation from Japanese rule in 1945, democratic education 
initiated by the U.S. military government (1945-1948) eventually 
afforded the populace an opportunity to access higher education. The 
common people who were able to access elite or higher education not 
only witnessed the granting of special socio-political rights to the 
Yangbans, but also recognized the instrumental values of education 
that endowed the upper class with socio-economic interests. Although 
the commoners were rarely allowed access to higher education on 
account of a rigid stratified social system, they desired to participate in 
traditional elite education or colonial higher education. Under the 
stratified social system, the ruling class, who were able to access 
higher education, was separated from the subordinate class. In spite of 
this divided system, the desire for education was different for the 
different social classes. Moreover, the rapid change of politics, 
economy, and society from the traditional bureaucratic Confucian 
society to the modern industrial democratic society necessitated the 
creation of human capital, thus promoting an academically oriented 
society. 

From the perspective of Korean cultural history, the 
contemporary high level of interest in education among the Korean 
people is the result of two significant factors: the opening up of higher 
education to more than a privileged minority; and the potential of 
education to confer upon its recipient both higher social status and 
economic success. The Korean people's desire for more education was 
a major factor in the expansion of the national higher education system 
as well as the development of the national economy. (Note 1 1) In 
particular, economic growth in the 1960s and 1970s was the result of 
the expansion of higher education. On the other hand, this enthusiasm 
for education has had a downside: excessive private education 
expenditures, social disharmony between the rich and the poor, 
promotion of an academic attainment-oriented society, and an 
examination hell for college entrants. In spite of these social and 
educational problems, the rapid growth of higher education leading to 
rapid economic development has come to be regarded as a model for 
the developed countries as well as developing countries. A 
fundamental cause of this economic and educational success was 
Koreans' desire for education rooted in a Confiician cultural, although 
national economic development policy played an important role as 
well. 

Without a correct understanding of this national enthusiasm for 






Vittn7/ftr>nfl fl<;n ftflii/enan/vQn97 html 



10/29/01 




EPAA Vol. 9 No. 27 Lee; The Establishment of Modern Universities in Korea 



Page 11 of 1 5 



education, it is meaningless to discuss the expansion of higher 
education and rapid economic development in contemporary Korea. 
Grasping the negative side of educational enthusiasm is no less 
necessary than understanding its positive aspects. 

Notes 

' Confucianism is an ideal ethical-moral system based on the teaching 
of Confucius, a Chinese philosopher in the sixth century BC. 

^ In the traditional period, Korean higher education fostered the elite 
who can lead the Korean people. Therefore, the words “higher 
education” would have different connotations than the ancient Greek 
or the medieval Western higher education. 

^ The Three Kingdoms were Koguryo (37 BC-AD 668), in the north; 
Packche (18 BC-AD 660), in the southwest; and Silla (57 BC-AD 
935), in the southeast. 

Choson society was classified into three classes: Yangban (the ruling 
class), Pyungmin (the common people), and Sangnom or Cheonmin 
(the lower people or the mean people). Generally, Chungin (the 
professional group) belonged to the common people. 

^ The Chou Dynasty of China divided social classes into four strata 
according to occupations: scholar, farmer, manufacturer, and 
merchant. Following these social strata, the Choson people respected 
scholars but despised manufacturers and merchants. Accordingly, the 
Yangbans and the Commoners ignored the two occupational groups. 

^ According to KNCU (1960), Severance Union Medical College was 
established in 1905 (p.l3). On the other hand. Son (1985) wrote 1904 
(?)(p. 70). 

’ Cf. Underwood (1926) wrote that Ehwa-hakdang opened in January, 
1886. Son (1985) mentioned that the school began with one girl in 
May, 1 886. Yu (1992) noted that Ehwa-hakdang began with a 
governmental official's concubine on May 31, 1886. Ehwa Woman's 
[Women's] College opened with fifteen students in 1910 (Adams, 

1965, p. 2; S. Lee, 1989, p. 89; Underwood, 1926, p. 1 13). 

* Cf. Yu (1992) wrote that H. G. Appenzeller began to teach English 
for two students on August 3, 1885 (p.49). 

’ In-soo Son (1985) describes chronologically the founding of 
missionary schools between 1885-1909 (pp. 76-77). In May, 1910, the 
entire number of authorized private schools in Korea was 2,250; and 
796 schools among them were established by the Western missionaries 
(S. Lee, 1989, p.90; Son, 1985, p.323). 

In 1999, Buddhists made up 45.7%, and Christians comprised 
51 .8% of the Korean believers. The number of believers was 
22,597,824 (50.7 percent of Korean population) (Ministry of Culture 
and Tourism, 1999). Although Confhcians only account for 0.9 
percent of Koreans, Confucianism remains a core culture in Korean 
society. 

" Between 1965 and 1996, the average annual growth rate of GNI 
(gross national income) reached about 8 percent, and GNI per capita 
increased from 105 to 1 1,380 US dollars (National Statistical Office: 
Seoul, Korea, 1999). In addition, between 1945 and 1999, Korean 
higher education increased from 19 schools, 1,490 teachers, and 7,819 
students to 354 schools, 55,718 teachers, and 3,154,245 students 
(Ministry of Education and Korean Educational Development 
Institute; Seoul, Korea, 1999). The total student population of higher 
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education expanded by 403 times. 
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Abstract 

This article reports the results of research into the career 
motivation and satisfaction of a sample of over 3,000 
teachers and school administrators in four countries: 
Australia, New Zealand, England, and the USA. Using the 
participants' own words, we explore the effects on 
educators of recent international educational change, 
understood here as a subcategory of more general social 
trends. Bourdieu's concepts of the Right and Left Hands 
of the state are used to interpret the experience of teaching 
in a climate where, while more is expected and demanded 
of schools, and schools and teachers are scrutinised as 
never before, educational resources have become scarcer, 
and the status and image of teaching as a profession has 
declined. 



Educational Change as Political Change: Consequences 
for Teachers and Administrators 

Belief that schools could and should address the challenges for 
nation states of globalisation and other trends, real and imagined, have 
become the motivation for radical changes to the management of 
education. The school reform movement itself is part of a larger 
political trend characterised by attempts to make nations, industries, 
companies Emd even individuals more competitive. Strategies adopted 
are well known but included various types of deregulation, and the 
restructuring of industries, with down sizing, rationalisation and cost 
cutting the order of the day. The social consequences of these changes 
have been profound and have included increases in unemployment and 
underemployment, combined with a switch to "flexible" modes of 
employment, such as short-term contracts or casual work. 

At the same time as these changes were being made, states 
increasingly have withdrawn from a number of sectors in which they 
traditionally had been involved and had supported, including public 
housing, social security, and education. The result has been to leave 
the newly under- or unemployed without a safety net and consequently 
even more unsupported and vulnerable. Social security has been 
replaced by widespread individual insecurity and an increase in social 
disruption and distress, and individual suffering has been the result. 

The Right and Left Hands of the State 

Bourdieu (1998) has written on these large and significant 
changes to the nature of the state. He refers to the institutions that have 
largely shaped and driven the reforms, that is cabinets, finance 
ministries and treasuries, and banks, as the Right Hand of the state. 
Those concerned with the public interest and welfare such as 
education, social services, public broadcasting, he calls the Left Hand. 

The Left Hand, including “family counsellors, youth workers, 
rank and file magistrates, and increasingly primary and secondary 
school teachers “ (p.2), finds itself confronted with the human 
consequences of economic restructuring. Members of the caring 
professions increasingly experience themselves as “sent into the front 
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line to perform so called social work to compensate for the 
inadequacies of the market” (p. 3). Bourdieu contends that the Right 
Hand does not know and does not want to know what the Left Hand is 
doing, and certainly does not want to pay for its activities. Social 
service agencies of all sorts are thus faced with the demand to do more 
with less. 

The antipathy of the Right Hand to the Left is also graphically 
illustrated in the decline of the status of those professions that perform 
the work of the Left. Included in this trend has been a decline in 
relative salaries for the Left. Bourdieu argues that “the salary granted 
is an unequivocal index of the value placed on the work and the 
corresponding workers” (p. 3). This decline has been paralleled by an 
increase in the prestige and remuneration awarded the Right Hand 
professions, all those who work with money rather than people. 

The Contradictions of High Modernity: Deregulation and 
Low Trust 

Another important aspect of economic and social restructuring 
also had its impact on teaching and teachers, and indeed education 
may be seen as the most extreme case of this trend, the trend to 
employ an auditing model of "quality control." 

Popkewitz (2000) noted that educational reform, as with other 
contemporary efforts at restructuring, is characterised by both 
centralising and decentralising tendencies. Decentralising/deregulation 
leads inevitably to potential loss of cetralised control and subsequent 
anxiety about maintenance of authority relations andmaintenance of 
quality. The phenomenon of the audit has emerged as one attempt to 
deal with the contradictions and uncertainty unleashed by these recent 
social and economic changes. 

Power (1994) in his book on the audit explosion discusses the 
expansion of this phenomenon. Originally a financial activity, auditing 
now takes a somewhat different form as it pervades other fields such 
as the environment, medical practice, and education. Paranthetically 
one might add that in a climate where the bottom line is the ultimate 
arbiter, the choice of a financial model of quality assurance is not 
surprising. 

According to Power, audits are designed to provide assurance 
and the abatement of risk, along with transparency of action, quality, 
value for money, best practice, and freedom from harm. He contends 
that the “fad” for auditing arose out of the contradiction that “on the 
one hand [there is] the need to extend a traditional hierarchical 
command in order to maintain existing structures of authority; on the 
other the need to cope with the failure of this style of control, as it 
generates risks which are increasingly hard to specify and control”, (p. 
6 ). 

The currently favoured model of auditing. Style A - Power 
maintains there are other available models - applies across disparate 
arenas. Central features of the model include long distance control, 
usually by external agencies, quantitative measures, low trust, and ex 
post control. These important features are linked. For instance, the 
involvement of outside bodies of experts in the oversight of activities 
has facilitated a shift in trust from operatives, the performers of 
activities, to auditors, those who police performance. Operatives are no 
longer to be trusted to do their jobs correctly, efficiently, effectively, 
and indeed ethically, but auditors are trusted to ensure that this all 
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occurs. 

The audit as it is currently conceived comes to shape the 
activities it is meant merely to oversee. If, for instance, counting is in 
favour, quantity over quality will prevail. If evidence of regulatory 
activity is required, regulation will increase and with it the associated 
paper work which is its evidence. Popkewitz (2000, p. 1 8) discussed 
how auditing in education systems performs that shaping function: "In 
this sense we can think of auditing as a way to 'reason' that has 
practical consequences. It shapes the conduct of professionals and 
organisations by asking that the standards of performance function as a 
technology to evaluate individuals. This is evident in systems of 
teacher education that focus on performance outcomes, as well as 
certain ways of thinking about students' learning. Auditing, then, is a 
knowledge that functions as an active intervention into organisational 
life, reshaping activities according to the norms of a fundamentally 
'opaque expertise.'" (Popkewitz, 2000, p. 18) 

In the field of education, the audit is epitomised by the British 
Office for Standards in Education (OFSTED) system of school 
inspections, but each national or state educational system has its own 
versions. One of OFSTED's main tasks has been to set up a system of 
school inspections, which occur every four years. Results of OFSTED 
inspections and national tests of student achievement are used to 
construct schools “leagues tables”. These tables are published in the 
“interests” of keeping “consumers' informed”. They also form the 
basis of decisions to intervene in schools or change their status, 
including the most radical intervention — closing the school down. 

Similarly, in the United States, the audit currently takes the form 
of state-designed testing programs. Currently most states mandate tests 
in some academic subjects and 21 states have plans to rate schools 
based on results of these tests. In addition, several districts are looking 
at the possibility of linking teachers' promotion and salaries to the 
performance of their students on these tests. The National Reading 
Panel authorized by the Reading Excellence Act of 1997, recently 
issued its findings and has called for the adoption of one particular 
approach to teaching beginning reading. (National Reading Panel 
Report: http://www.nationalreadingpanel.org/Documents/default.htm) 

Consequences for Teachers of Social and Political Change 

The International Teacher 2000 Project was launched to 
investigate the consequences for teachers and administrators of the 
changes to education systems described above. Increasingly, it has 
become obvious that the factors that influence teachers' occupational 
satisfaction are no longer confined to the microcosm of the school 
(Sergiovanni, 1967, following Herzberg, et. al., 1959). Instead the 
“Third Domain” (Dinham and Scott, 2000) has a major influence in 
determining how teachers feel about their work. 

Whereas older models of occupational satisfaction posited two 
spheres for discussing satisfaction and dissatisfaction, the actual work 
of teaching and the conditions under which work must be performed, 
the new theory proposes a three- factor model. The Third Domain, 
encompasses factors at the system level, as well as wider social forces. 
As Bourdieu would argue, these include the increase in social 
disruption and suffering attendant upon economic rationalisation, and 
the decrease in respect, recognition and reward given to professions 
forming the Left Hand of the state (1998). 
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The researchers of the Teacher 2000 team sought to cast their 
net wider than an investigation of “teacher stress” and instead set out 
to investigate what motivates teachers, and what satisfies and 
dissatisfies them about their work. To date over, 3000 teachers in four 
countries (Australia, England, New Zealand and the USA) have been 
surveyed using parallel forms of a self-report instrument. Results have 
been remarkably consistent and have been reported in detail elsewhere 
(see Dinham and Scott, 1996; Scott, 1999; Harker et al., 1998, for 
completed reports of national phases of the research). 

Teachers in all four countries were found to be motivated by a 
desire to work with and for people, and to “make a 
difference” (Dinham and Scott, 2000) by assisting children and young 
people to reach their potential, experience success, and grow into 
responsible adults. Teachers everywhere found high satisfaction in this 
aspect of their work. In all four countries satisfaction remained high on 
a small focused set of “core business” aspects of teaching. This 
satisfaction occurred at the personal levels of working directly with 
children: experiencing success with pupils/students, working 
cooperatively with other members of the education community, and 
professional competence/development (Dinham and Scott, 1996; 

Scott, 1999; Harker et al., 1998). However, they rated their overall 
occupational satisfaction as low, and many find themselves more 
dissatisfied later in their careers than when they began teaching. 

Levels of dissatisfaction were not uniform across all aspects of the 
work, however. 

Aspects of teaching associated with school level factors — school 
climate, leadership, resources, and reputation — were rated more 
ambivalently. Considerable variation was based, not surprisingly, on 
the school in which the individual teacher was currently employed. 
Aspects of the work that caused teachers dissatisfaction were more 
numerous and varied somewhat from country to country and according 
to current local issues and problems. As an example, when the 
Australian survey was in progress, a long-standing pay dispute, and 
previous, unpopular changes to promotion procedures, conflict over 
which was still occurring, led to considerable discontent and industrial 
unrest, reflected in both the numeric ratings on relevant questiormaire 
items and the comments made by teachers (Dinham and Scott, 1996). 
For English teachers, the National Curriculum and OFSTED 
inspections were major issues, again registered in responses to the 
survey (Scott, 1999). 

Despite national variations, there was also a core of Third 
Domain issues that concerned all teachers regardless of residence. 

They included decrease in status and recognition of the profession, 
outside interference in and de- professionalisation of teaching, pace 
and nature of educational change, and increase in workload. Mean 
ratings on items concerned with these issues were universally low, and 
observations about enforced change, increased outside interference in 
education, increased “non-core” workload, and low pay and status 
formed the majority of comments on open-ended sections of the 
questionnaire. 

Previous publications have explored the quantitative measures 
of discontent including the development of the scales used to assess 
satisfaction and dissatisfaction with facets of teaching and its context 
(Scott, Dinham and Brooks, 1999). This paper will focus on the 
participants' own words used to describe their experience of teaching 
in an era of profound and enforced social and educational change. 
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In Teachers* Own Words 

In this section we use the teachers' own words to illustrate and 
support the points we have made, above. We have left the teachers' 
choice of words and modes of expression in the forms we received 
them. 

The satisfactions of teaching 

The main satisfiers of teaching are well known and have been 
documented by many researchers. As we note above these are the 
satisfaction of working with children and seeing them achieve, 
working collaboratively with other members of the education 
community, and achieving personal professional growth. Teachers in 
this study have continued to confirm these areas as satisfiers. 

Working with and assisting others. A major and universal 
satisfier is the opportunity to work with children and with other 
members of the educational community. Teachers from all countries 
listed this aspect as a main source of satisfaction. 

I enjoy children and being with children. I find being a 
team member satisfying, working towards achieving goals 
together. NZ classroom teacher, 53. 

I enjoy working with children I enjoy helping people. I 
enjoy working with Teachers College students. NZ 
classroom teacher, 48. 

Joy of seeing children progress. Relationships with 
parents. Teamwork as a result of collaborative planning. 

UK head teacher, 46. 

Contact with children. UK, head of dept, 39. 

Team work of the staff. Working with other head teachers. 

UK, head teacher, 51. 

I have a dynamic coordinator of special ed (my dept). A 
lot of good things are being done for our department and 
our spec, ed students. She is definitely a child advocate. 

The morale is high and we feel like a family (we were 
very close and communicated a lot) everyone knows what 
is going on and issues that come up can be discussed 
freely. USA, specialist teacher. 25. 

Professional efficacy and making a difference. A supreme 
satisfaction of teaching was the opportunity to “make a difference”, 
contribute to a young person's development, and to see the results of 
that contribution. This for many or most teachers IS teaching and the 
externally imposed demands for assessment, record keeping, and 
accountability are in comparison “nonsense”. They interfere with 
teaching, or to quote an NZ teacher, change it to “being an accountant 
and not an educator”. 
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I enjoy making a difference to people's lives. UK, head 
teacher, 46. 

Teaching was a career path that I chose so that I would be 
able to help people and hopefully enable them to achieve 
the best that they can for themselves. Working in what is 
considered a "disadvantaged" school, doesn't bother me at 
all, in fact the challenge is more rewarding. Aust 
classroom teacher, 22. 

Evidence of learning and creativity. Pupils growing up 
and turning out all right after all. UK, head of dept, 49. 

Satisfaction comes... when you can sit down at a break and 
say you've taught a lesson well (emphasis in original). NZ 
classroom teacher, 37. 

Knowing my class is happy to be at school. Giving a 
boost of confidence or highlighting a child's achievement 
who does not normally do well. NZ classroom teacher, 36. 

I love to watch the growth in my students as they 
experience success throughout their educational 
endeavours. It pleases me to my very soul that I am able 
to help a child become an independent learner. It's like 
being a parent a multitude of times each year . . .1 love it! 

USA classroom teacher. 

I teach because I love to work with students and 
experience the 'high' that comes from watching them 
achieve success with my help. It is the one driving force 
that keeps me in the profession. USA, classroom teacher, 

43. 

I love feeling I have really helped and inspired someone. I 
enjoy planning my lessons and seeing kids succeed. USA, 
classroom teacher. 53 

Professional challenge and growth.Teachsrs from all countries 
commented on aspects of teaching work that allowed for both the 
utilisation of personal qualities such as flexibility, creativity and the 
ability to respond well to challenge and the opportunity to continue to 
grow and develop as an individual. 

The scope for being creative, improvising and exploring 
different ways to teach/meet students' needs. That every 
day is different- bringing new surprises/challenges. The 
classroom is a very vibrant place to be. NZ classroom 
teacher, 30. 

I like teaching because it allows me to continue to learn 
and grow in many ways. USA classroom teacher, 58. 

The adventure of learning — both pupils and teacher. UK, 
specialist, 52. 
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Creativity of producing teaching materials. UK, classroom 
teacher, 29. 

The joy of overcoming difficulties. UK classroom teacher, 

48. 

I think to be a good and successful teacher you need to be 
able to make your own decisions, devise your own 
philosophy and keep yourself motivated and buoyant. 

Aust classroom teacher, 50. 

Dissatisflers: Dealing with the social consequences of change 

To those on the inside, the teachers and adminstrators 
themselves, the pressures to which teaching has been subject as a 
profession may feel unique. An exploration of the nature of current 
social change, however, makes it clear that these demands are 
manifestations of tendencies in the larger society. Teaching may thus 
be seen as a case study of the effects of these tendencies on professions 
of the Left Hand. The Teacher 2000 Project marks the first time the 
pressures from the education system and societal levels have been 
documented and fully recognized. 

The effects of social disruption. As discussed above, Bourdieu 
has noted that the many consequences of the international waves of 
change and restructuring are frequently characterised by an increase in 
unemployment, material inequality, and a variety of ills that flow from 
them. Professionals who form the Left Hand of the state deal with the 
consequences of these ills for individuals, families, and communities. 
There are at least two major aspects of how this expansion of 
responsibilities has negatively affected teachers’ occupational 
satisfaction: 

1 . Dealing with social disruption has widened the scope of 
teachers' work to make teachers, in the words of one NZ 
practitioner “counsellors, social workers, nurses”, and in an 
Australian teacher's “bouncer, child counsellor, animal trainer, 
army sergeant, school nurse, megaphone (not the operator the 
actual machine)“. The increase in work has both an indirect - 
“kids are so needy these days”, to use one US teacher's words- 
and a direct cause as schools have become increasingly seen as 
the appropriate agencies to deal with a large and proliferating 
array of social problems. 

2. Seeing the harm done to individuals is in itself an affront to 
many members of a profession dedicated to nurturing 
individuals and seeing them do well. 

Teachers from all countries commented on both aspects, the 
increase to their own responsibilities, and the pain and frustration of 
seeing children's life chances compromised by social circumstances. 

I am interested and always have been in teaching my 
subjects, but I find it almost always a struggle battling 
with students' lack of real interest and maturation as well 
as general and severe behaviour problems associated with 
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the above reasons or due to welfare problems. Teachers 
cannot do all things: teach, counsel, and to perform 
administrative duties (which are always on the increase) 
and counselling or reprimanding. Severely influence 
teacher satisfaction. Aust classroom teacher, 55. 

Poor family backgrounds — lack of experiences, language, 
attendance at school, physical/emotional abuse, all factors 
which severely affects children's progress. NZ, specialist 
reading teachers, 41. 

We are now expected to deal with barriers to learning by 
contacting health and social services, getting more 
involved in what goes on in the child's life outside school. 

NZ classroom teacher, 30. 

What the survey does not bring up is the type of students 
we must face (unlike the classrooms of yesteryear). Terms 
like BD, LD, ADD, ADHD, lEP, etc, make me feel I 
deserve a psychologist's salary instead. USA classroom 
teacher, 48. 

Lack of sufficient number of counselors - kids are very 
"needy" today. Social workers are scarce in the city, 
county and usually ineffective. USA classroom teacher, 

55. 

Teachers are always addressing student and parent 
welfare. Very little is ever done for teacher welfare. 

Teachers are holding society together but if in-servicing 
money etc is not forthcoming for teacher welfare, there 
will be a breakdown in the education system. Aust 
classroom teacher, 30. 

The tendency of the Right Hand of the state to cut back on 
funding in areas of social need (while demanding more) has resulted in 
growing teacher dissatisfaction due to the compromises it forces in 
education, as well as the damage wrought elsewhere in society which 
it expects schools and teachers to rectify. 

Lack of funds to help all children. NZ classroom teacher, 

44. 

Lack of up-to-date resources, particularly for special 
needs children. NZ, specialist, 44. 

No money appears to be available for children who 
desperately need help. UK classroom teacher, 42. 

Student welfare is another big problem area. There is not 
enough support for students with severe psychological and 
behavioural difficulties. Before one can contemplate 
special placement (as scarce as hens' eggs) the fellow 
students and staff often are put under enormous stress. 

Children should not have to put up with these students in 
great need to the detriment of their health and education. 
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Conversely we must provide for the students who are 
reacting to other impossible home situations. Aust 
classroom teacher, 49. 

Erosion of professionalism. Erosion of professionalism also has 
at least two aspects: 

1 . Lowering of the status of and respect for the profession, 
symbolised for many teachers by the relatively low pay the work 
is awarded. 

2. Erosion of the scope for exercising professional judgement, 
independence, and competence and of the time to do “real 
teaching”. 

The lack of trust in the professionalism of teachers and anxiety 
about national educational standards have led to a policing mentality 
among administrators, a tendency noted across many domains in the 
widespread move towards the adopting of Style-A auditing. The 
consequence has been an anxiousness to standardise and document all 
aspects of the work, lest quality be compromised by leaving too much 
to the judgement of practitioners. The introduction of many more 
reporting and documenting requirements, as well as the standardisation 
of many aspects of teaching, contributes both to the much noted 
increase in overall work load and to the erosion of the sorts of 
pleasures of the job described above, i.e., flexibility, challenge, 
creativity, working with and for people. These two facets of the 
erosion of professionalism, increased work and decreased respect, 
were summed up by one NZ teacher as “constant demands and 
negative comments”. 

Status, criticism, recognition and salary. 

Teachers need to be respected by other teachers, parents, 
students and the whole wide world. Respect and money! 

USA classroom teacher, 48 

Teaching isn't like it used to be and the money isn't worth 
the abuse we cop day in day out. Aust classroom teacher, 

32. 

The status of teachers must be raised in regard to their 
place and respect in society. In order to effectively 
educate and care for children we must be respected, have 
status and held in HIGH esteem. Raise salaries - it's a 
start. USA classroom teacher, 49. 

Lack of recognition for experience and skills, constant 
denigration of skilled staff. UK, classroom teacher, 40. 

I also feel considerably underpaid. There are few perks to 
the job, if any. When I compare myself and people in 
industry I feel particularly cross, especially since I am 
better qualified than many of them. I feel that unless 
people are really committed then they should not enter the 
teaching profession these days. UK classroom teacher, 39. 
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The press always seems to be hammering teachers. UK 
classroom teacher, 49. 

Total lack of respect for teachers. NZ, classroom teacher, 

52. 

Lack of pay parity, poor salary. NZ, specialist, 44. 

Issue of teacher status/parent — community relations and 
media perceptions seem to me to be key inter-related 
issues. Balance of good community relations seems to be 
slipping away - schools subsequently have been under a 
lot of parent criticism. Aust classroom teacher, 49. 

Over the last 6 years I have become less satisfied with my 
chosen career due to the ever-increasing workload, never 
ending changes huge responsibilities, and constant media 
bashing. I paid my way through 8 years of study, bought 
many excellent resources, and have given up my health 
and quality of life to receive very little recognition or 
thanks. Aust classroom teacher, 28. 

Erosion of professionalism/professional practice, increased 
paperwork. The flip side of the erosion of professionalism for many or 
most teachers is the increasing intrusion and interference by those 
education administrators, politicians, the press, school governors who 
know “naff all” about teaching, to quote an English head teacher. 

Teachers gain little respect...! feel more like a slave than 
an educator. Aust, classroom teacher, 27. 

Erosion of professionalism — we are completely 
emasculated by the national Curriculum/OFSTED/targets. 

U K head of dept. 

Schools become the "meat in the sandwich" during 
elections. Politicians use Education as a political football. 

Those in power beat their chests about "reforms" they 
have achieved and those wanting power assault us with 
what they will do to make teachers work more efficiently 
and produce improved student outcomes. To listen to their 
drivel on the TV and radio an ordinary person would think 
that teachers totally lack intelligence and the professional 
will to direct their own activities towards improved 
outcomes for the students. Aust, classroom teacher, 36. 

Teachers feel disillusioned with teaching because the DSE 
[Department of School Education, now the Department of 
Education and Training] shows lack of leadership. They 
are only interested in cost cutting measures. They bring in 
changes without any consultation with the people that 
matter - the teachers. They are out of touch with reality. 

Aust classroom teacher, 42. 

Rubrics must be made for every assignment, teacher 
judgement not valid, [leaving teaching] I will really miss 
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the children and TEACHING (which I believe I am not 
being allowed to do with all this NONSENSE- "show a 
rubric for everything you display" . . . "show how this 
lesson teaches a MAP skill" [state mandated test]. USA 
classroom teacher, 50. 

I found over the years the amount of preparation and 
documentation and accountability and paper work has 
increased until I find I must consciously say that is all I’m 
going to do tonight/this weekend - as 1 must spend time 
with my family and friends. Aust classroom teacher, 41. 

Unrealistic expectations of top administration. Increased 
load of meaningless paper work. I feel that education in 
general and my district in particular is responsive to 
"trends" in education. Whatever is the latest issue 
becomes our focus. I feel that our top-level administrators 
are very out of touch with what goes on in the classroom 
on a daily basis. I don't think my principal or 
superintendent could survive a week in my job. But they 
are constantly pushing for what they perceive as 
improvement, while only making my job harder. 
Classroom teachers are bombarded with paper work. We 
spend so much time on useless paperwork that planning, 
evaluating, and teaching time are seriously impacted. 

USA classroom teacher, 49. 

Spending time on things that have no benefit to the 
children I teach and are not important to me apart from 
keeping my job. NZ classroom teacher, 27 

Shocking admin work, copious assessment etc details. NZ 
classroom teacher, 50. 

Paperwork — the endless evaluation/appraisal that no one 
else is interested in reading but which must be filed. NZ, 
classroom teacher. 

A parent of a child I taught 9 years ago has just told me 
that her child has just been accepted for King's College, 
Cambridge to study medicine, and thanked me for my 
ability to develop and encourage his interest in science 
and maths. This was just pre National Curriculum, and pre 
all the 1001 "new" initiatives. A moral here, I think. I no 
longer have the time to do the same. UK Dept Head, 50 

I can’t help but feeling as a person who is prepared to 
give/care (and I generally think most teachers are like 
this) that I am being “used" by the system employing me, 
because each year I seem to be giving a little more (at the 
expense of my family and personal hobbies, etc). Aust, 
classroom teacher, 28. 

I am convinced that the 60/70 hours per week required to 
do "my job"- in the holidays too, has been a significant 
factor in my illnesses. Over the last few years I have also 
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suffered 2 serious episodes in which a key factor is the 
overbearing and never easing demands of the Principal's 
role. Aust principal, 54. 

Quotes from UK teachers summed up well the ways that these 
various pressures are eroding some of the core satisfiers of teaching 
(viz,, facilitating student achievement, helping others, and one's 
professional growth) with professional autonomy and judgement being 
replaced by machine-like routines: 

I am very concerned at the increased stress levels being 
experienced by teachers. I joined this profession 24 years 
ago and felt I contributed more to children's education 
because I had time to relate to the children I taught. Now I 
am under so much pressure to reach standards I have little 
time to really talk to the pupils. I feel more like a machine 
as the years go by with little time for reflection. UK 
classroom teacher, 46. 

Teachers feel like puppets; other people pull our strings. 

There is little vision left in the teaching profession - it's 
been weeded out over the last 10 years (and is still being 
weeded out). UK classroom teacher. 

In addition to the emphasis on external assessment and 
standardised testing compromising the opportunity for professional 
practice, teachers also feared that it would distort the entire 
educational enterprise 

I greatly fear that the net effect of this standards 
movement is an increase in the gap between the learned 
and unlearned, and the subsequent "lowering of the bar," 
which is the last thing we said we would ever consider, 
and the first thing that my district thought about (but, did 
NOT do), when they received back the latest round of 
writing scores. USA classroom teacher. 

There is too much teaching of programs rather than 
teaching of children in our district. The goal of education 
should not be to look good but to do whatever it takes to 
reach all students. If we did that, looking good would take 
care of itself USA classroom teacher. 



Erosion of Professional Relationships. Attempts to change the 
way schools are managed has had unfortunate consequences for 
collegial relations, and more so in those countries, notably Britain, 
where these have been the most far-reaching. Emphasis on a more 
managerial style for head teachers (principals) and the devolving to 
them of more discretion — and responsibility — over matters such as 
pay and promotion has frequently disrupted within-school relations. 
The requirement that principals implement changes that neither they 
nor their staff support has also had deleterious effects. 

As a bit of idealistic "old timer" veiy disappointed in new 
ethos of self-promotion, point-scoring, impressing others. 

Not as much openness, collegiality, sharing as when I 
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began. Some new principals seem to see themselves as 
CEOs managers, etc. Replaced at higher levels of D.S.E., 
senior officers flit from position to position. Where is the 
accumulated body of knowledge and experience? 
Dissipated in manageralism?? Aust, classroom teacher, 

49. 

Heads of schools playing one staff member off against the 
other. NZ classroom teacher, 39. 

Today's schools allow more room for personality 
clashes — ^you only get on by the word of others and not 
your ability, or potential. NZ, classroom teacher, 41. 

Head Teachers have no incentive to listen to staff. Greater 
Head Teacher powers make it impossible for teaching 
staff to have a professional voice. UK classroom teacher, 
30. 

The philosophy and practices of teaching have changed 
markedly from being collegiate and cooperative to be 
divisive and competitive. The principal has created a 
culture of distrust and rivalry between teachers and 
faculties. Many teachers are now perceiving undermining 
of their colleagues, plagiarizing programs, stealing 
resources as a means to get on with their careers. Aust 
classroom teacher, 35. 

Being required to implement changes in which I don't 
believe with a staff who also disagree with them is not 
motivating. Knowing that the doubts I and many of my 
colleagues have will be dismissed as cynical, progressive 
(which I am not) or a pathetic justification of failure, 
undermines my professionalism and educational 
experience. UK head teacher, 48. 

Decision-making is limited to executive staff in the 
school. 

Opportunities are limited to executive staff Aust 
classroom teacher, 32. 



Conclusions 

A consideration of teaching and its discontents may be seen as a 
case study of the effects of the current dominance of the Right Hand 
on professions of the Left. That is, a study of teachers' views of their 
occupation clearly shows the consequences for professions that deal 
with persons and their welfare in a climate where the bottom line has 
come to dictate the shape and nature of institutions and the relations of 
individuals to these and to each other. 

Teachers everywhere enter the profession to serve children. 
While they are, in general, pleased with their choice of career as it 
relates to working directly with children who are willing to participate 
and learn, outside forces have intervened to prevent teachers from 
performing their jobs as they perceive them. The result has been a 
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major decline in professional satisfaction. The increasing economic 
and social problems that teachers confront, combined with the efforts 
to have educational systems provide solutions for those problems, has 
led to an increase in the everyday work of teachers and its complexity. 

Students who are extremely emotionally and socially needy and 
who have serious self-discipline problems increase the pastoral 
demands of teaching. In addition, the expansion of external assessment 
requires the production of more written documents in greater detail, 
causing the increase in paperwork of which teachers complain. As 
Bourdieu predicts would be the case, teaching has also suffered a 
decline in respect and status, combined with reduced salary and 
resources available with which to do its work. 

We would therefore contend that the profound dissatisfaction 
expressed by teachers in all four countries is caused by the concurrent 
juxtaposition of and antithetical nature of two major factors: 

1 . Motivation to enter teaching. Teachers are motivated by 
altruism and activism in the sense of a desire to make a 
difference by aiding individual children. Teaching is an activity 
of the Left Hand, the welfare arm, of the state, and as such deals 
with the consequences of social change/disruption wrought by 
the Right Hand. It also attracts the antipathy of the Right Hand 
as reflected in its decline in pay, status, and recognition. 

2. The issue of control. Growing attempts to control the process of 
teaching in order to control its output supposedly to benefit the 
nation economically have increasingly taken the form of attacks 
on teacher professionalism. This has led to a decline in the 
opportunity to experience satisfaction with one's own 
professional activity and ensuing erosion of overall satisfaction. 

The melancholy conclusion to be drawn from this argument is 
that teaching is not and cannot be quarantined from the social context 
in which it is embedded. No amount of positive thinking or number of 
ringing admonitions can alter the effects on the profession of the 
general, profound decline in respect for and trust of those do people 
work. Similarly, working smarter or any number of other fashionable 
solutions cannot ameliorate the “intensification” (Hargreaves, 1994) of 
the work of teaching attendant upon these changes. What is required is 
a wider perspective on the nature and the enormity of the social 
changes that manifest at the “chalk face” in the patterns of discontent 
the voices of our participants reveal. 

Note 

The research reported here was supported by grants from UWS 
Nepean, The NSW Teachers' Federation, NSW DET, Massey 
University, The Nottingham Trent University and Rowan University. 
The support is gratefully acknowledged. 
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Resumen 

El articulo analiza las principales caracteristicas de los 
programas de maestria en ciencias sociales en la 
Argentina y evalua en que medida estos se constituyen en 
una estrategia innovadora frente a un entomo institucional 
que opone fuertes restricciones al cambio. La 
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investigacion se basa en un estudio de diecinueve casos de 
maestrias de ciencias sociales en universidades publicas y 
privadas, contando ademas con informacion cuantitativa 
de aquellas que solicitaron acreditacion en la CONEAU 
(Comision Nacional de Evaluacion y Acreditacion 
Universitaria) en 1998. Los programas son analizados en 
terminos de sus objetivos, insumos, procesos y productos. 

El trabajo concluye examinando las restricciones que 
limitan un desarrollo adecuado de los posgrados 
academicos y profesionales en el campo de la ciencias 
sociales. 

Abstract 

The article describes the main features of Argentine Social 
Science Masters programs and evaluates the extent to 
which they are developing innovative strategies in the 
present conservative institutional context. The research 
was based on nineteen case studies of Masters in public 
and private universities. Quantitative data were also 
gathered from those Masters that were accredited by the 
National Commission of Evaluation and Accreditation 
(CONEAU). All these data are analyzed in terms of the 
program’s objectives, inputs, processing and output 
indicators. The paper concludes with an analysis of the 
restrictions that limit the suitable development of 
academic and professional postgraduate courses in 
Argentina. 

1. Introduccion 

En el contexto de la expansion de la matricula de los estudios de grado 
y de crecimiento y diferenciacion institucional, los cursos de maestria 
ingresaron a escena en la Argentina en los anos ochenta, irrumpiendo 
en el molde tradicional europeo de licenciaturas de larga duracion y 
doctorados escasamente estructurados y desarrollados, prosperando 
con mayor rapidez y entusiasmo en las ciencias sociales. Teniendo por 

referenda este proceso expansivo, se llevo a cabo en el CEDES ' una 
investigacion sobre las maestrias de ciencias sociales, teniendo como 
fuentes la base de datos provista por la Comision Nacional de 
Evaluacion y Acreditacion (CONEAU), instancia que acredita 
obligatoriamente al conjunto de los posgrados y la realizacion de 

diecinueve estudios de caso en universidades publicas y privadas 

Ademas de caracterizar descriptivamente el universo bajo analisis, ha 
sido el principal interes de esta investigacion evaluar en que medida 
estos nuevos programas de posgrado se constituyen en una estrategia 
innovadora frente a un entomo institucional conservador, que opone 
fuertes restricciones al cambio. Esta empresa innovadora esta, empero, 
condicionada externa e intemamente para alcanzar con exito su 
proposito. En primer lugar, existen restricciones importantes en la 
disponibilidad de recursos humanos (docentes con credenciales de 
posgrado, grupos consolidados de investigacion, actividades de 
tutoria), financieros (fondos institucionales para la actividad de 
posgrado y para la realizacion de proyectos de investigacion, becas, 
bibliotecas) y fisicos (disponibilidad de infraestructuras y equipos). 

En segundo lugar, las condiciones internas a la institucion (las 
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universidades) y a las unidades academicas (las facultades, los 
departamentos) conforman un marco organizacional con escasa 
capacidad de procesar sin conflictos cambios en la estructura de 
funcionamiento y en la gestion. En general, las universidades 
latinoamericanas tienen graves dificultades para encauzar las 
demandas de innovacion que se generan en las bases disciplinarias. 
Finalmente, el marco regulatorio generado por las poh'ticas 
gubemamentales no siempre produce las senales apropiadas para 
acompanar estos cambios irmovadores, impidiendo la consolidacion de 
nuevas practicas de ensenanza e investigacion. 

Pensar en mecanismos innovadores en el nivel de posgrado supone 
adentrarnos en distintos modelos posibles de producir conocimiento y 
de capacitar a las futuras generaciones en esta actividad. La vision de 
la ciencia tradicional y sus mecanismos de organizacion, 
financiamiento y evaluacion es ya muy estrecha para abarcar la 
complejidad de los programas y funciones que brinda la formacion 
avanzada en la educacion superior. Desde esta perspectiva, hemos 
delineado un marco analitico que nos permitiera reflexionar sobre la 
diversidad hoy reinante y que guiara nuestra lectura de los casos y del 
material recopilado. Es por ello que, previo al analisis de la 
informacion que surge de la investigacion, brindaremos una primera 
reflexion teorica sobre esta cuestion. 

2. Condiciones ideates de funcionamiento de los posgrados 
academicos y aplicados 

Una tension claramente presente en los modelos de posgrado de la 
Argentina, expresada tanto en los objetivos y organizacion de los 
programas como en la evaluacion de la calidad de los mismos, es la 
que se presenta entre el modelo de posgrado academico y el modelo 
aplicado. Es claro que cada uno de estos modelos requiere de 
condiciones extemas e intemas de funcionamiento para lograr el exito 
en la prosecucion de los objetivos de los programas y para encauzar 
innovadoramente esta oferta. Un camino para analizar estas 
condiciones es utilizando los esquemas conceptuales que se esbozan 
en trabajos como los de Clark (1993 y 1995), Conrad et al. (1993) y 
Gibbons (1998). 

Tanto Clark como Conrad analizan los estudios de nivel de posgrado, 
centrandose, el primero, en las condiciones extemas e intemas que 
favorecen la integracion entre investigacion, docencia y aprendizaje. 
En otras palabras, Clark sera quien asuma la defensa del modelo 
academico o de la ciencia tradicional (tambien llamada modalidad 1 
del conocimiento por Gibbons). Por su parte, Conrad, en respuesta a la 
flierza arrolladora de la demanda de las maestrias profesionales, ira 
delineando ciertas condiciones innovadoras dentro de lo que 
podriamos llamar el posgrado aplicado. La vision de Gibbons es a la 
vez mas general e integradora de ambos enfoques. Si bien su 
preocupacion no son los estudios de posgrado sino las nuevas 
modalidades de produccion y distribucion del conocimiento asi como 
los cambios organizacionales que las mismas demandan, arroja 
tambien luz sobre el desarrollo future de los posgrados. En estos, el 
eje no sera la produccion y la formacion dentro del marco de las 
disciplinas ni una vision estrecha de corte profesional, sino el contexto 
de aplicacion o de resolucion de problemas (lo que Gibbons denomina 
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"modalidad 2" de produccion de conocimiento). A partir de los 
trabajos de Conrad y Gibbons es posible, entonces, delinear algunas 
condiciones extemas e internas al funcionamiento de los posgrados 
aplicados que resultan diferentes a aquellas pensadas dentro del 
enfoque de Clark para el modelo academico. Realizando un paralelo 
con la idea de Clark del nexo investigacion-docencia-aprendizaje, 
diremos que este nuevo modelo, esbozado impllcitamente en el trabajo 
de Conrad y mas abiertamente en el de Gibbons busca la integracion 
entre la resolucion de problemas o aplicacion, la docencia y el 
aprendizaje. Seguidamente brindaremos una sintesis de los principales 
aportes de estos autores que seran de utilidad para el analisis de 
nuestro objeto de estudio. 

2.1. Fuerzas de fragmentacion e integracion del nexo 
investigacion-docencia-aprendizaje 

En una de sus ultimas obras, Clark (1995) analiza las principales 
experiencias intemacionales de estudios de posgrado, para 
desarrollar — a partir de dicho estudio — un conjunto de categorias 
analiticas apropiadas para identificar y explicar la presencia de un 
nexo “virtuoso” entre investigacion, ensenanza y aprendizaje. La 
integracion de estos tres pianos de acciones se alcanza gracias a ciertas 
condiciones externas (en el piano de la organizacion de los sistemas de 
educacion superior) e internas (en el nivel de la institucion 
universitaria y sus unidades operativas, tales como los departamentos). 
Sin embargo, si bien a los ojos de Clark este nexo “virtuoso” encuentra 
su modelo ejemplar en las graduate schools de las universidades de 
investigacion norteamericanas, en general el nexo es amenazado 
permanentemente por fuerzas de fragmentacion. Las cuatro tendencias 
que contribuyen a separar la actividad de investigacion y la educacion 
universitaria en los niveles avanzados son: la masificacion de la 
educacion superior, la creciente demanda de expertos profesionales 
para el mercado de trabajo, la brecha entre el conocimiento de frontera 
y el conocimiento codificado, pasible de ser ensenado en la 
universidad, y el aumento de la supervision y control de los gobiernos. 
Estos factores promueven el corrimiento en algunos casos hacia la 
investigacion {research drift) y en otros hacia la ensenanza {teaching 
drift), aislando una de otra labor. Se da entonces el caso de 
investigadores que se dedican exclusivamente a esta actividad en 
laboratories, centres de investigacion, instituciones del gobiemo o de 
la industria; docentes que ensenan sin incorporar los nuevos avances 
que se logran en el campo de la investigacion y alumnos que se forman 
sin contacto con la practica cientlfica. 

El vertiginoso crecimiento de la matricula en el nivel superior ha 
propiciado la expansion del sistema y con ella su diferenciacion 
institucional, favoreciendo el surgimiento de instituciones de 
educacion superior solo dedicadas a la ensenanza. El sistema se torna 
entonces cada vez mas “postsecundario” y menos “superior”. Esto no 
solo tiene lugar en el nivel del grado sino que se extiende a la 
educacion mas avanzada. En el nivel del posgrado esto se manifiesta 
en el crecimiento de aquellos programas que tienen menor sustento en 
la actividad de investigacion como es el caso de las maestrlas. En 
Estados Unidos se otorgaron en 1994 387 mil tltulos de maestria, el 84 
por ciento de los cuales correspondla a aquellas de corte profesional, 
siendo las mas importantes las maestrlas en negocios (MBA) y 
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educacion (para los docentes), y 41 mil titulos de doctorado (LaPidus, 
1997). 

Por su parte, la cada vez mayor especializacion de los temas de 
investigacion demanda una creciente concentracion de fondos, equipos 
y personal especializado que es dificil de retener en los espacios 
tradicionales de docencia y aprendizaje. En aquellos casos en que estas 
unidades se ubican en las universidades, lo mas probable es que sus 
integrantes esten exceptuados de realizar actividades docentes. En gran 
parte de los pai'ses analizados por Clark, la investigacion se ha 
expandido por fuera de las universidades, en organismos publicos, 
militares, industrials e instituciones sin fines de lucro. 

Finalmente, la separacion del financiamiento de la ensenanza y de la 
investigacion impulsada en los ultimos tiempos por los gobiemos es 
un elemento adicional que propicia la separacion del nexo. Por un 
lado, los gobiemos buscan disminuir el costo de los graduados 
universitarios fomentando la expansion de instituciones volcadas mas 
centralmente a la ensenanza que a la investigacion. Por el otro, los 
fondos de investigacion se distribuyen segiin mecanismos 

competitivos reforzando el efecto Mateo: se les da mas a los mejores . 
Un ejemplo de ello es el caso del Reino Unido. 

En sintesis, los sistemas nacionales de educacion superior, sujetos a 
una fuerte demeinda social y con recursos publicos que no crecen a la 
par, invierten en instituciones dominadas por la ensenanza, en los 
niveles iniciales mas masivos y en programas de posgrado no 
orientados hacia la investigacion. La hipotesis de Clark es que estos 
son procesos sin retorno. A pesar de ello, considera que es posible 
constatar la presencia de condiciones que tiendan a contrarrestar estas 
fuerzas disgregadoras del nexo investigacion-docencia-aprendizaje. 

Las condiciones que propician, segun Clark, la integracion del nexo 
son tanto externas como internas a las universidades. Distingue asi tres 
niveles de condiciones propiciatorias: en el nivel del sistema nacional 
de educacion superior, en la organizacion de la institucion universitaria 
y en la unidad de ejecucion o unidad operativa (departamento). 

En el piano extemo, en primer lugar, la diferenciacion de instituciones 
permite resguardar un espacio donde este nexo es posible. En palabras 
de Clark: 

Si un sistema moderno de educacion superior ha de dar 
sustento y proteccion a la integracion de la investigacion 
con la docencia y el aprendizaje, debera en el nivel macro 
desarrollar ima concentracion institucional del nexo. Si se 
difunde entre todas las instituciones, el nexo se tornara 
muy costoso, con fondos insuficientes en el nivel de las 
unidades, y debilitado por la diflision. (Clark, 1995, pp. 

214-215, nuestra traduccion) 

Una segunda condicion que favorece en el piano de las condiciones 
externas el fortalecimiento del nexo es la presencia de competencia 
entre instituciones y programas. La administracion descentralizada del 
financiamiento lleva a la competencia entre programas e instituciones 
por lograr obtener fondos para su unidad. A diferencia de la 
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competencia en el piano empresarial donde el objetivo es maximizar la 
ganancia, en el ambito de la educacion superior la competencia se da 
en el piano de la reputacion. Esta descansa a su vez en disponer de 
investigadores de alto nivel academico. La posibilidad de atraer dicho 
personal depende de poder mantener condi clones de trabajo adecuadas 
para este. Estas condiciones y un personal de alto nivel conducen a im 
aumento de la productividad en la investigacion, lo cual resulta en que 
la institucion sea un iman para la investigacion y la capacitacion de los 
estudiantes mas talentosos. Una vez que la reputacion se ha 
establecido, esta es el principal active del que dispone la institucion 
universitaria. 

La tercera condicion en el piano del sistema es la presencia de una 
ideologia de la unidad del nexo. Esta creencia descansa en el ideal 
humboltiano de “educacion a traves de la ciencia” donde en ambitos 
institucionales tales como el laboratorio o el seminario de 
investigacion, docentes y alumnos entran en una relacion intima de 
mentor y aprendiz, instancia en la cual no solo se transmite 
conocimiento tangible sino tambien tacito sobre la practica de 
investigacion. 

La ultima condicion integrativa es el mecanismo de financiamiento. 

La separacion del financiamiento de la ensenanza y la investigacion 
conduce, en ultima instancia, al corrimiento hacia la ensenanza o 
hacia la investigacion. For tanto, lo mejor es que las universidades 
perciban el aporte publico a traves de un subsidio global, siendo la 
propia institucion la que redistribuye los fondos entre ensenanza e 
investigacion. 

En el piano de la universidad, tres elementos ayudan a la integracion 
del nexo: 1) la diferenciacion de la ensenanza de grado de la ensenanza 
de posgrado a traves de una estructura organizacional particular: la 
escuela de posgrado; 2) la autonomia en la actividad de investigacion, 
gracias a la diversificacion de las fuentes de financiamiento y la 
presencia de subsidies cruzados entre el grado y el posgrado y 3) la 
competitividad alcanzada por la ocupacion de nichos, por la escala y el 
alcance de los estudios de posgrado. 

El ultimo componente esencial para que el nexo entre investigacion- 
docencia-aprendizaje sea viable es la presencia de ciertas condiciones 
en la unidad operativa o ejecutiva. Nuevamente el modelo 
norteamericano de la organizacion departamental resulta el ambito mas 
propicio, segiin Clark, para que un grupo de docentes - investigadores 
transmita no solo conocimiento tangible a sus alumnos sino tambien el 
conocimiento tacito necesario para embarcarlos en la actividad de 
investigacion. El departamento academico deviene el ambito adecuado 
para fusionar el grupo de ensenanza y el grupo de investigacion. A lo 
largo de un periodo de cuatro anos (si llevan adelante estudios de 
doctorado) los estudiantes estan expuestos a este conocimiento tacito 
asi como tambien a informacion y tecnicas transmitidas por los 
docentes. Mientras que la organizacion departamental hace posible la 
transmision del conocimiento tangible, el grupo de investigacion es el 
principal vehiculo para la socializacion del conocimiento tacito. 

La fuerza del modelo de la escuela de posgrado norteamericana para 
garantizar la presencia de este nexo ha llevado a su difusion en el 
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piano internacional. A principios de los anos noventa se comenzaron a 
crear escuelas de posgrado en el Reino Unido, Holanda, Alemania 
(Burgess, 1997) y son el modelo implicito de muchos posgrados 
latinoamericanos. 

Como es posible apreciar, en las mismas fuerzas fragmentadoras se 
hallan imph'citamente los caminos para alcanzar la integracion. 

2.2. Hacia un modelo alternativo de produccion del conocimiento: 
el nexo entre resolucion de problemas -docencia - aprendizaje 

Si bien el esquema conceptual que elabora Clark es un buen punto de 
partida para evaluar las condiciones que hacen posible la tan deseada 
integracion entre la investigacion, la docencia y el aprendizaje, es 
limitado si se busca tambien examinar los requisites para que la 
abundante oferta hoy existente, orientada hacia la capacitacion y 
formacion aplicada para el sector publico y el privado sea de calidad, 
eficiente y pertinente. Tal como lo destacan Conrad et al (1993), el 
“silencioso exito” del posgrado norteamericano se expresa en una 
proliferacion de programas, alumnos y graduados en estas 
orientaciones profesionales. For el contrario, las maestrias vinculadas 
mas intimamente con la investigacion son consideradas o bien solo un 
estadio previo hacia el doctorado o, en el peor de los casos, un premio 
consuelo para aquellos que no han logrado completar con exito el mas 
alto nivel. 

Si tenemos en cuenta que la mayor parte de los graduados en el nivel 
del posgrado no se van a desempenar en tareas de docencia e 
investigacion dentro del sistema de educacion superior sino que lo 
haran en otras instituciones del sector publico y privado, seria 
esperable que las condiciones propiciatorias de un nexo virtuoso entre 
el saber aplicado, la docencia y el aprendizaje fueran diferentes de 
aquellas que son exitosas para el posgrado academico tradicional. El 
conocimiento tangible y tacito transmitido a traves de los posgrados 
deberia responder a esta situacion. Esta es una realidad incluso para 
algunos graduados del nivel de doctorado. 

El modelo tradicional anglosajon de master vinculado con el PhD. y, 
por tanto, con la investigacion, se contrapone tambien con el nuevo 
modelo — mas profesional y aplicado — en el perfil de sus estudiantes. 
Cada vez mas los estudiantes de posgrado son adultos con edades 
superiores a los 30 anos y con dedicacion parcial al estudio. En estos 
programas, se enfatiza la practica mas que la teoria, la adquisicion de 
destrezas, mas que la investigacion, y el entrenamiento mas que el 
trabajo academico (Conrad et al, 1993). 

Este modelo de maestria fue acompanado por el uso de nuevas 
tecnologias de ensenanza y por formatos no tradicionales en la 
organizacion de los cursos. En 1987 en los Estados Unidos, por 
ejemplo, cerca del 70 por ciento de los titulos de maestria otorgados no 
requirieron la realizacion de tesis (Conrad et al, 1993). Del trabajo de 
campo realizado por el estudio de Conrad et alsurge tambien como 
relevante la presencia de un cuerpo de profesores donde se incluyen 
profesionales con experiencia especialmente en el mercado de trabajo, 
actividades de ensenanza-aprendizaje participativas y con practicas en 
los ambitos laborales donde se desempenaran los graduados (practicas 
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en laboratorios, trabajos de campo, pasanti'as en organismos del sector 
publico y privado, etc.), adecuados sistemas de recompensas para los 
docentes, orientados a premiar la actividad desarrollada en estos 
posgrados y mecanismos de asistencia economica a los estudiantes de 
tiempo parcial. Estas condiciones se alejan en parte de los atributos de 
calidad que suelen acompanar al modelo tradicional de posgrado de 

investigacion^. 

El modelo de posgrado que describe la investigacion de Conrad se 
acerca al paradigma de Gibbons (1998) sobre las nuevas tendencias en 
la educacion superior. La tesis principal de Gibbons es que las 
universidades estan organizadas segiin las estructuras de las disciplinas 
cientificas (lo que Gibbons denomina modalidad 1) y que estas 
estructuras se estan modificando en direccion a lo que pasa a llamar 
modalidad 2. En el espacio de las universidades, esta modalidad 2 
supone que la produccion y divulgacion de conocimiento — la 
investigacion y la ensenanza — ya no son actividades autonomas, que 
se llevan a cabo en instituciones aisladas, sino que implican una 
interaccion con otros diversos productores de conocimiento. El punto 
de partida de su marco analitico es que las consecuencias de la 
masificacion de la educacion superior y las presiones internas de la 
competencia ban contribuido a modificar la base sobre la que se realiza 
la investigacion y la produccion de conocimiento en general. A partir 
de esta afirmacion marca las diferencias entre la modalidad tradicional 
de produccion de conocimiento, la cientifica o modalidad 1 , y la nueva 
0 modalidad 2. A cada una de estas modalidades le corresponderan 
ideas, metodos, valores y normas diferentes, asi como estructuras 
organizacionales, fmancieras y de control de calidad tambien diversas. 

Dentro de la modalidad 1 , las practicas de investigacion y ensenanza 
se organizan segiin la estructura de las disciplinas y bajo el supuesto de 
que la especializacion es una forma segura de hacer avanzar el 
conocimiento. La estructura de las disciplinas da la pauta a los 
investigadores sobre cuales son los problemas importantes, como se 
los debe abordar, quien ha de hacerlo y que se considera una 
contribucion a esta esfera. En los aspectos sociales tambien prescribe 
las reglas para acreditar a los nuevos investigadores, los 
procedimientos para seleccionar a los docentes y los criterios para su 
avance en la vida academica. La investigacion que cumple con las 
normas estipuladas por las estructuras disciplinarias es considerada 
“ciencia” y a sus hacedores “cientificos”. Para buscar entonces otra 
denominacion que identifique la actividad y a sus hacedores en la 
modalidad 2 Gibbons propone “conocimiento o investigacion” y 
“ejecutantes o investigadores”. No se sugiere, empero, que los 
ejecutantes de la modalidad 2 no se comporten de acuerdo a las 
normas del metodo cientifico. 

En contraste con la modalidad 1 , la modalidad 2 se organi 2 a en 
funcion de una aplicacion o problema particular. Con este proposito, 
un equipo de investigadores desarrolla un consenso teorico — que 
trasciende el marco disciplinario — ^para resolverlo. Para ello crean 
estructuras teoricas, modelos, metodos de investigacion y formas de 
practicas propios. El esfuerzo que realizan es acumulativo pero su 
contribucidn no es necesariamente a una disciplina especial. La 
modalidad 2 tiene la caracteristica de la multidisciplinariedad. Esta 
caracteristica se asocia con el desplazamiento de los intereses de los 
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cientificos desde la ciencia unificada hacia el estudio de las 
propiedades de los sistemas complejos. La aplicacion de la modalidad 
2 en los planes de estudio exige pasar de un aprendizaje basado en las 
disciplinas a otro basado en los problemas. En particular, los 
estudiantes de posgrado deben poder trabajar creativamente en equipos 
integrados por personas con muy distintos antecedentes cientificos y 
deben saber como manejar mas de un marco intelectual y como 
relacionarlo al problema de investigacion en manos. Considera central 
las asociaciones con otras instituciones y ve con buenos ojos los 
nuevos arreglos o convenios que se alcanzan en el piano de la 
educacion transnacional. El aprendizaje basado en los problemas debe 
incluir tambien capacitacion en elaboracion de modelos y simulacion 
por computadora. 

La comunicacion de los resultados en la modalidad 2 no se limita a las 
publicaciones en revistas especializadas sino que se emplean multitud 
de formas electronicas, organizacionales, informales, redes de 
comunicacion, etc. 

Al estar centrada en la resolucion de problemas, aumenta el niimero de 
lugares donde puede crearse este conocimiento. Las universidades son 
entonces solo uno de estos lugares posibles. Participaran tambien de 
esta actividad, institutes de educacion superior no universitarios, 
centres de investigacion, organismos piiblicos, laboratories 
industriales, consultoras, etc. Para poder llevair adelante con exito la 
modalidad 2, las universidades deberan convertirse en instituciones 
porosas, mas abiertas y dinamicas en la busqueda de alianzas y 
asociaciones de lo que son actualmente. 

En el piano de la estructura organizacional interna, pone en duda a la 
institucion universitaria y al departamento como el eje integrador. 
Debido a la especializacion y a la fragmentacion que ban acompanado 
a la division y subdivision del conocimiento, las facultades se ban 
convertido en categorias de organizacion mas que en categorias 
intelectuales. Incluso los departamentos son simples unidades 
administrativas y no tanto centros intelectuales. El curso y el equipo de 
investigacion ban devenido en la verdadera unidad academica. Los 
pequenos equipos universitarios de investigacion son, por su parte, 
vulnerables a la gran movilidad de los jovenes investigadores, atraidos 
por nuevas ofertas extemas al ambito academico, pero su fiierza 
radica en las redes de conocimiento a las que tienen acceso a fuerza de 
la competencia y la flexibilidad con que pueden abordar nuevos 

problemas 

La investigacion en la modalidad 2 requiere nuevas formas de 
financiamiento. No depende tanto de los fondos del gobiemo o de las 
fundaciones sin fines de lucro, sino de las empresas, los organismos 
piiblicos y los grupos de presion afectados directamente por los 
problemas a analizar. En tal sentido, un porcentaje creciente de las 
investigaciones se realizan en programas especificos fmanciados por 
organismos externos con fmalidades definidas. 

El control de calidad es tambien diferente. En la modalidad 1, la 
calidad se determina esencialmente por la evaluacion colegiada de las 
contribuciones que efectiian los individuos. El control se mantiene 
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mediante una seleccion cuidadosa de las personas que se consideran 
competentes para actuar como pares, lo que a su vez se determina por 
las contribuciones que hayan efectuado a su disciplina en el pasado. En 
la modalidad 2 se agregan criterios adicionales dados porque el 
contexto de aplicacion ahora incorpora una gama diversa de intereses 
intelectuales as! como otros de indole social o politica. Surgen 
entonces preguntas del tipo: la solucion a la que se arriba tras la 
investigacion, ^sera competitiva en el mercado?, ^sera eficaz en 
funcion de sus costos?, <;,sera aceptable socialmente? En otras palabras, 
en la modalidad 2, la evaluacion se centra en la eficacia, eficiencia y 
pertinencia de los productos logrados. 

Del analisis emprendido por los estudios de Conrad et al y de la 
prospectiva que elabora Gibbons sobre los caminos futuros de la 
produccion de conocimiento, surgen una serie de condiciones extemas 
e internas que contribuyen a integrar el nexo aplicacion-docencia- 
aprendizaje. 

Desde el punto de vista externo, a la diferenciacion de instituciones 
dentro de la educacion superior senalada por Clark, se le agrega la 
multiplicacion de ambitos institucionales dentro y fuera del sistema de 
educacion superior donde se produce conocimiento y la constitucion 
de alianzas estrategicas entre ellos. En tal sentido, la competencia no 
se remite solo a la reputacion adquirida por los recursos humanos 
(docentes y alumnos) disponibles en la institucion, sino que se amplia 
para abarcar la capacidad de resolucion de problemas por parte de 
estos centres de investigacion dentro y fuera de las universidades. El 
conocimiento tangible y, en particular, el conocimiento tacito no se 
transmite solo en el grupo de investigacion sino que se obtiene 
tambien a traves de practicas en los lugares de trabajo y a traves de 
pasantias. El fmanciamiento va dirigido especialmente a la formacion 
de analistas o reconfiguradores del conocimiento y a investigaciones ' 

destinadas a resolver problemas. En tal sentido, en la delimitacion del 
problema y en los resultados, participan las multiples instancias 
interesadas. Como vemos. Gibbons trata aqui de acentuar las nuevas 
tendencias de vinculacion de la investigacion con la industria. 

En terminos de las condiciones internas, de acuerdo al paradigma que 
perfila Gibbons, no seria suficiente separar la formacion de grado de la 
de posgrado, reservando para esta ultima la funcion de produccion de 
nuevo conocimiento, sino que el grado tambien tendria que ir 
transformandose, rompiendo los moldes disciplinarios. Este punto no 
queda muy claro en su trabajo. Desde nuestra perspectiva, todo parece 
indicar que la multidisciplinariedad se alcanza mejor en el posgrado 
que en el grado. 

La reforma en el nivel de los estudios de grado y de posgrado que 
propone Gibbons radica en otorgarle a los programas y a los tltulos 
mayor flexibilidad, a la par que se imparten nuevos valores que 
establecen las bases de la aceptacion del cambio permanente y de la 
necesidad de un aprendizaje de por vida. 

Finalmente, frente a la importancia que Clark le otorga al 
departamento y al grupo de investigacion como unidades basicas de 
ejecucion, tanto Conrad como Gibbons hacen hincapie en la relevancia 
del trabajo grupal para la resolucion de problemas en un ambito 
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multidisciplinario. De acuerdo a Gibbons, el departamento es solo un 
ambito administrativo, ya no mas intelectual. Los distintos grupos que 
vienen trabajando alrededor de un problema, traspasan los llmites 
estrechos de su disciplina en pos de comprender los sistemas 
complejos que enfrentan. 

El marco analltico desarrollado hasta aqui nos permitira a continuacion 
organizar el estudio de las condiciones internas y extemas del 
funcionamiento de las maestrias en ciencias sociales de la Argentina. 
Para ello, comenzaremos describiendo sinteticamente sus rasgos 
principales para abordar luego un examen mas detallado de las mismas 
aplicando el concepto de funcion de produccion. 

3. Estructura y funcionamiento de las maestrias de 
ciencias sociales 

En las ultimas decadas, a las ciencias sociales les corresponde la 
mayor expansion del sistema, tanto en numeros de programas como en 
matricula. En el nivel de posgrado, 32 por ciento de los titulos 
ofrecidos por las universidades estatales y privadas corresponde a esta 
area de estudio, con un claro predominio de los programas de maestria 
(Trombetta, 1999). En particular, las ciencias economicas 
(administracion y economia) son las que mayor presencia tienen 
dentro de este tipo de programas, siguiendole en importancia la 
sociologia y las ciencias politicas (ver Cuadro N° 1). 

Cuadro N" 1 

Titulos de Maestria por disciplina en las Ciencias Sociales. 
Rep. Argentina, 1997 (en porcentajes) 

Fnente: Trombetta (1999) 



Disciplina 


Porcentaje 


Administracion y Economia 


CM 

UJ 

bo 


Sociologia 


15.4 


Ciencia Politica 


13.6 


Derecho 


7.7 


Comunicacion 


5.3 


Demografia 


2.4 


Relaciones Piiblicas 


1.2 


Otras 


0.6 


Total 


100 


Numero de titulos 


169 
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En cuanto a la matricula, el 43 por ciento de los 3 1 mil alumnos del 
nivel de posgrado cursa programas de maestn'as, el 70 por ciento de 
ellos en universidades estatales (ME, 1 999 b). 

Con el fin de profundizar en el conocimiento del funcionamiento de 
estos programas, se decidio emprender una serie de estudios de caso y 

examinar los datos disponibles en la CONEAU al respecto^. El 
conjunto de la informacion producida y recopilada por este medio file 
entonces analiza dentro del marco analitico antes desarrollado, 
ordenando las condiciones externas e intemas de funcionamiento a 
partir de aplicar el concepto de fimcion de produccion a este campo de 
estudio particular. Mas alia de las limitaciones de este concepto para 
explicar los complejos procesos sociales, politicos y economicos que 
tienen lugar en el ambito educativo, consideramos que es de utilidad 
para organizar la informacion que surge precisamente de los 
indicadores empleados desde un piano de analisis estrictamente 
organizacional. Comenzamos asi por examinar los objetivos de los 
programas, para despues adentrarnos en el estudio de sus insumos, 
procesos y productos educativos. 

3.1. Objetivos 

Un punto de partida para analizar la coherencia interna de estos 
programas de posgrado es examinar cuales son sus objetivos. Segun 
surge de la lectura de los folletos de las maestrias de ciencias sociales 
y de las entrevistas a los directores de los programas, podriamos 
distinguir dos conjuntos de tipos de objetivos: academicos y 
aplicados. Dentro de estos ultimos distinguimos entre aquellos 
orientados a la obtencion de una credencial profesional y aquellos 
orientados a la resolucion de problemas. En la mayor parte de los 
casos estas orientaciones no se presentan nunca en forma pura, pues en 
todos esta presente la formacion profesional, especialmente por las 
deficiencias, en cantidad y calidad, de puestos en el mercado de trabajo 

academico^. Pero tambien en todos encontramos cierta orientacion 
hacia lo academico, pues existe una real necesidad de formar docentes 
de posgrado en las ciencias sociales. Es probable que el origen de este 
aumento de la demanda de credenciales academicas haya sido el 
resultado de senales emitidas por la politica publica al requerir titulos 
del mas alto nivel para la actividad de docencia universitaria. Las 
chances de que el posgrado sea fmalmente acreditado por la CONEAU 
o que reciba fondos competitivos de programas como el FOMEC 

(Fondo para el Mejoramiento de la Calidad Universitaria)^ aumentan 
en la medida en que el plantel de docentes tiene mayor nivel de 
formacion academica. 

A pes 2 u de que la mayoria de las maestrias de ciencias sociales 
presenta multiples objetivos, es posible distinguir un predominio mas 
claro de lo profesional- credencialistico en las de administracion de 
empresas; de lo academico, en las de economia, sociologia y ciencias 
politicas y una orientacion hacia la resolucion de problemas en las de 
corte multidisciplinario. En estas ultimas, los planes de estudio estan 
orientados hacia la produccion de conocimiento sobre las propiedades 
de sistemas complejos, como el estado y las politicas piiblicas, y no 
exclusivamente a una disciplina en particular y estan, en el sentido de 
Gibbons (1998), guiadas por un contexto de aplicacion y con una 
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finalidad pragmatica. 

3.2. Insumos 

Con relacion a los alumnos de las maestrias de ciencias sociales, los 
datos existentes permiten inferir que se trata mayoritariamente de 
adultos, con edades que promedian los 34 anos y que trabajan (Garcia 
de Fanelli, 2000). 

Para caracterizar al cuerpo docente de las maestrias en ciencias 
sociales utilizamos dos indicadores construidos en base a la 
informacion provista por la CONEAU. Los mismos son el grado 
academico maximo y la productividad en termino de publicaciones. 

Con relacion al titulo maximo alcanzado por los docentes, debemos 
tener presente que, segun lo determina la Ley de Educacion Superior 
sancionada en 1995 (Ley N° 24.521, articulo 36), los docentes 
universitarios de todas las categorias deben poseer titulo universitario 
de igual o superior nivel a aquel en el cual ejercen la docencia. 
Tratandose entonces de programas de maestria, los docentes deberian 
tener titulo de magister o doctor. 

Siguiendo este criterio, en la columna 3 del Cuadro N° 2 se puede 
apreciar la proporcidn de la planta docente que, en cada tipo de 
posgrado y sector, no cumple con esta normativa. En promedio, esta 
proporcion asciende al 37 por ciento, con escasas diferencias entre el 
sector de universidades piiblicas y privadas. Dado el reciente 
desarrollo del nivel de posgrado y, en particular, del nivel de 
doctorado en ciencias sociales en la Argentina, esta proporcion no es 
sorprendente De todos modos, la situacion difiere particularmente 
segun disciplinas. Dentro de las maestrias en economia, la proporcion 
de docentes que, como maximo solo detenta el titulo de grado es casi 
inexistente. Son, por otro lado, las que cuentan con una mayor 
presencia de docentes con doctorado. En el extreme opuesto ubicamos 
a los docentes de las maestrias de administracion, un tercio de los 
cuales posee como titulo maximo el de grado . 

Cuadro N" 2 

Republica Argentina. Distribucion porcentual de los 

docentes 

de las maestrias de ciencias sociales segun grado 
academico maximo en 1998 
(Fuente: CONEAU, 1999) 



Programa 


1 .Titulo 
de 

Grado 


2. 

Especialista 


3= 

1+2 


Magister 


Doctor 


TOTAL 


20.0 


17.3 


37.3 


25.9 


36.7 


Sector Estatal 


17.1 


21.1 


38.2 


24.0 


37.9 


Sector Privado 


26.0 


9.9 


35.9 


29.5 


34.5 


ADMINISTRACION 


28.2 


12.8 


41.0 


29.2 


29.9 


Sector Estatal 


24.9 


16.1 


41.0 


25.4 


33.7 
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Sector Privado 


30.8 


10.1 


40.9 


32.3 


26.8 


ECONOMIA 


1.8 


13.3 


15.1 


22.6 


62.4 


Sector Estatal 


2.8 


16.2 


19.0 


26.8 


54.3 


Sector Privado 


0.0 


8.3 


8.3 


15.5 


76.2 


Ciencias 

POLITICAS 


11.5 


26.3 


37.8 


22.8 


39.6 


Sector Estatal 


11.4 


28.6 


40.0 


22.8 


37.3 


Sector Privado 


12.3 


12.3 


24.6 


22.8 


52.7 


OTRAS 

Ciencias .Sociales 


17.0 


21.6 


38.6 


22.9 


38.4 


Sector Estatal 


16.9 


22.3 


39.2 


22.9 


37.9 


Sector Privado 


18.2 


4.5 


22.7 


22.7 


54.5 



En nuestros estudios de casos se observa ademas que una alta 
proporcion del cuerpo docente de las maestrias de economia tiene 
formacion de posgrado en el exterior, principalmente en universidades 
norteamericanas. Aunque en menor proporcion, este patron tambien es 
comiin entre los profesores de ciencias politicas. 

Con relacion a la productividad de los docentes medida en terminos de 
la cantidad de publicaciones, se puede apreciar que, en promedio, cada 
docente ha publicado cuatro articulos en revistas intemacionales y 
nacionales con referato, dos libros y tres capitulos de libros. La 
productividad promedio de cada docente es en total de 29 trabajos 
escritos, entre las distintas posibilidades (ver Cuadro N° 3). 

Cuadro N“ 3. 

Republica Argentina. Promedio de publicaciones por docentes, 
sectores y disciplinas en los programas de maestria 
en ciencias sociales en 1998 
(Fuente: CONE AU, 1999) 



Disciplinas 


Rev i Stas 
sin 

referato 

argent. 


Revistas 

con 

referato 

argent. 


Rev i Stas 
sin 

referato 

extranj. 


Rev i Stas 
con 

referato 

extranj. 


Libros 


Capitulos 
de Libros 


Anales de 
Congresos 


Arts . de 
divulgacidn 


Total 


1 




1.0 


am 


1.4 


1.4 


1.3 


1.9 


9.3 


23.0 


2 




1.4 


6.9 


5.3 


1.6 


2.6 


3.4 


3.9 


28.4 


3 


6.3 


am 


7.2 


5.1 


4.0 


6.4 


6.6 


11.7 


50.0 


4 


4.1 


1.7 


3.7 


3.5 


2.5 




2.9 


5.6 


28.4 


Total 


am 


1.4 


3.9 


2.9 


2.1 


3.0 


3.1 


8.2 


29.0 



1: Administracion; 2: Economia; 3: Ciencias Politicas y Relaciones Intemacionales y 4: 
Otras Ciencias Sociales. 
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De estos datos es posible tambien inferir que cada comunidad 
disciplinaria tiene canales privilegiados para transmitir los nuevos 
conocimientos. Asi, los docentes de administracion y los de ciencias 
politicas producen un mayor niimero de articulos de divulgacion que el 
resto. La produccion de libros es por su parte mas importante en las 
ciencias politicas y en la sociologia que en economia o en 
administracion . 

Sin duda un elemento clave que explica la cantidad y calidad de los 
recursos disponibles es la disponibilidad de fondos. En tal sentido se 
observa que la principal fuente de fondos de las maestrias es lo 

recaudado en concepto de aranceles a los estudiantes^. El monto 
promedio pagado en tal sentido es de seis mil dolares, siendo mucho 
mas elevado en el sector privado que en el publico y en las ciencias 
economicas que en las otras ciencias sociales (Garcia de Fanelli, 

2000 ). 

En consecuencia, casi todo el fmanciamiento del costo de los estudios 
de posgrado recae sobre los estudiantes y sus familias. La institucion 
universitaria piiblica (via subsidio del gobiemo federal) financia 
centralmente la infraestructura basica y pone a disposicion del 
programa la biblioteca de la facultad o del departamento donde se 
asienta formalmente la carrera. 

Otra limitacion en la disponibilidad de recursos es la falta de un 
programa amplio de becas o creditos educativos. Si bien tres de cada 
cuatro maestrias en ciencias sociales que se presentaron a la CONEAU 
afirman disponer de becas para sus estudiantes, en los estudios de 
casos se observa que las mismas son sumamente escasas y que, a lo 
sumo, cubren parcialmente el total del arancel (Garcia de Fanelli, 
2000). Elio es una de las razones que explican que la mayoria de los 
estudiantes sean de dedicacion parcial. 

Tambien son escasos los recursos financieros disponibles intemamente 
para la investigacion, tanto aquellos provenientes del sector publico 

como del sector empresario'*^. Esta es una restriccion importante para 
que se desarrollen y consoliden grupos de investigacion, ambitos 
privilegiados para transmitir el conocimiento tacito de esta actividad. 
Actualmente los alumnos tienen su primer contacto con la tarea de 
investigacion al momento de realizar la tesis. Ante el temor que la 
misma despierta, postergan indefinidamente su presentacion, aun 
cuando hayan concluido de cursar y aprobar todas las materias y 
afrontado los altos costos de este proceso de aprendizaje. 

Finalmente, el financiamiento del posgrado — sobre la base casi 
exclusiva de los aranceles que abonan los estudiantes — si bien brinda 
recursos suficientes como para remunerar a los profesores, limita el 
poder de decision de sus autoridades y lo toma altamente vulnerable 
frente al ciclo economico. Como el posgrado es sostenible 
financieramente siempre y cuando se logre mantener una masa critica 
de alumnos, ello impide el desarrollo de planes de estudios flexibles, 
que contemplen materias optativas, y la planificacion de actividades 
de mediano plazo, pues se carecen de bases firmes para llevarlas 
adelante. 
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3.3. Procesos de ensenanza -aprendizaje- investigacion y gestion 

El analisis de los planes de estudio y las entrevistas realizadas revelan 
una gran ausencia: espacios donde los alumnos participen activamente, 
junto a los docentes, en actividades de investigacion. Esto es 
importante especialmente en los posgrados academicos, con miras a 
articularse con flituros estudios doctorales, aunque no deja de serlo 
tambien en aquellos orientados hacia la resolucion de problemas. En 
este ultimo caso, la actividad podria estar orientada al diseno de 
modelos que faciliten una mejor comprension de las propiedades de 
los sistemas complejos que abordan. En particular, los estudiantes de 
posgrado deberian poder trabajar en equipos integrados por 
profesionales con diversos antecedentes cientificos y profesionales y 
aprender como manejar mas de un marco intelectual para relacionarlo 
con el problema de investigacion 

En el caso de los posgrados profesionales y los orientados a la 
resolucion de problemas en campos disciplinarios, faltan tambien 
espacios de capacitacion en los lugares de trabajo, tales como 
realizacion de pasantias. Existen, por supuesto, algunos casos que ban 
avanzado significativamente en esta direccion, incorporando practicas 
novedosas tales como trabajos de campo realizados en estrecho 
contacto con el sector productivo, tesis definidas en funcion de 
problemas planteados por sectores publicos y organismos no 
gubernamentales, pasantias en distintas areas del sector publico, etc. 
Tambien estos programas ban tratado de firmar convenios con 
universidades extranjeras y locales a fm de establecer alianzas 
provecbosas para los alumnos de la maestria, gracias al intercambio de 
docentes, alumnos y el otorgamiento conjunto de titulos (Garcia de 
Fanelli, 2000). 

En cuanto a la duracion de las maestrias, en promedio demandan dos 
anos para el cursado de las materias, lo cual esta dentro de estandares 
internacionales. En los paises industrial izados existen tambien ofertas 
de un ano de duracion, pero exigen al alumno una dedicacion de 
tiempo completo. En las maestrias argentinas, empero, parece 
excesivamente extenso el plazo que se otorga para la realizacion de la 
tesis. Mas de la mitad de los programas fijan un plazo superior a los 
dos anos. Como minimo, por tanto, una maestria puede llegar a 
demandar cuatro anos, lo cual puede ser excesivo si se piensa que lo 
deseable es que aquellos que se orienten por el mercado academico 
realicen tambien estudios doctorales. Probablemente lo extenso de este 
plazo se explique, en primer lugar, porque no existen becas que 
permitan que los alumnos se dediquen exclusivamente a esta actividad, 
con el agravante de que, tras fmalizar el cursado de las materias, 
pierden contacto con sus docentes y solo estan inmersos en su 
actividad laboral cotidiana. En segundo lugar, como es la primera vez 
que los alumnos entran en contacto con la actividad de investigacion, 
requieren de un periodo de aprendizaje antes de saber claramente 
como orientarse en esta practica. En tercer lugar, puede ocurrir que las 
autoridades de las maestrias fijen exigencias superiores a las normales 
para una tesis de este tipo, confundiendo la tesis de maestria con una 
de doctorado. Finalmente, falta un sistema estructurado de supervision 
que logre encauzar mas prontamente la realizacion de las tesis. 
Reconociendo este problema, algunos planes de estudio contemplan la 
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realizacion de talleres donde se discuten los proyectos de tesis al poco 
tiempo de iniciarse la cursada. 

Respecto de la gestion, uno de los aspectos mas iimovadores y 
relevantes es su manejo altamente descentralizado y el papel del 
mercado en la coordinacion de las poHticas. Las autoridades del 
posgrado son las que en definitiva toman las decisiones cruciales sobre 
el plan de estudio, la admision de los estudiantes, la seleccion y 
remuneracion de sus docentes. Este es un aspecto importante pues les 
otorga mas flexibilidad y plasticidad para el cambio y la 
experimentacion de nuevas practicas organizacionales y pedagogicas. 
Las particularidades &Qeste nivel se explica por el codigo genetico 
impreso en el memento de su gestacion. Fruto de la masificacion de la 
ensenanza del grade, necesidades de mayor capacitacion y procesos de 
aculturacion (importacion del modelo anglosajon de ’dentro 

de una estructura de carreras de grade que responde al modelo de 
Europa continental), los posgrados experimentaron un crecimiento 
explosive desde mediados de los ochenta. Como surgieron en un 
entomo caracterizado por la restriccion de recursos financieros, se 
debieron autofinanciar a traves del cobro de aranceles a los 
estudiantes. Esta necesidad de autofinanciacion y su escasa 
importancia dentro de la arena de la politica universitaria, a la par que 
limita su desempeno, los torna mas libres y autonomos frente a la 
inercia burocratica de la institucion. La inclusion de los posgrados dio 
lugar asi a un cambio profundo en el piano organizacional, sin 
todavia lograr su articulacion adecuada con el nivel de grade. 
Funcionan con una logica totalmente diferente a la de este: los estudios 
de posgrado son arancelados, los docentes son incorporados a traves 
de procesos de seleccion que suele quedar a discrecion de los 
directores de los programas, el nivel de las remuneraciones se fija en 
forma descentralizada — no solo dentro de cada facultad, sino en cada 
programa — y se implementan procedimientos de seleccion de los 
alumnos mucho mas selectivos que en el grado. 

La fuerte diferenciacion que media entre el grado y posgrado en su 
aspecto organizacional podria constituir una condicion propicia paia 
crear un ambito institucional donde sea posible resguardar el nexo 
entre investigacion- ensenanza -aprendizaje . En los hechos, este nexo 
es practicamente inexistente en el nivel de grado y solo se concentra en 
unos pocos campos disciplinarios como las ciencias basicas, algunas 
carreras de humanidades y las ciencias agrarias 

Por otra parte, a diferencia del grado, en los estudios de posgrado 
existe competencia por captar alumnos, docentes y por la reputacion. 
Tal como ftmeionan actualmente, en los posgrados de ciencias sociales 
prima una logica de coordinacion de mercado. Existe un mercado 
donde se compra y vende servicios educativos, particularmente porque 
su financiamiento principal descansa en el cobro de aranceles a los 
estudiantes. La intervencion del estado, subsidiando a los alumnos a 
traves de becas, es, como ya senalamos, escasa. En este contexto, los 
posgrados compiten por captar alumnos y, en algunos casos, esta 
competencia entre programas de un mismo nicho de mercado da por 
resultado que no se logre reunir una masa cn'tica de alumnos como 
para poder autofinanciar el programa. En algunas maestrias del sector 
privado, por ejemplo, para solucionar este problema se implementa 
una politica de subsidies cruzados. 
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La coordinacion casi exclusiva del mercado en terminos del 
financiamiento impone, empero, serias restricciones a la expansion de 
estos posgrados, especialmente en lo que respecta a la actividad de 
investigacion que los mismos pueden llevar adelante. En el caso de la 
investigacion basica, por ser esta un bien publico, es logico que exista 
una subinversion si su financiamiento descansa exclusivamente en el 
mercado. La expansion de la investigacion aplicada requiere a su vez 
del financiamiento de organismos piiblicos y privados interesados en 
los resultados de la misma. 

En su faz positiva, esta misma logica de mercado torna a las 
autoridades mas preocupadas por la satisfaccion de los estudiantes. 

Asi, es una practica hoy ampliamente difundida en el posgrado, no asi 
en el grado, que los alumnos evaluen a sus profesores. 

3.4. Productos de los programas 

Una aproximacion, todavia muy imperfecta, a la medicion de la 
eficiencia de los estudios de posgrado en la Argentina permite arribar a 
la conclusion de que el nivel de rendimiento de las maestrias de 
ciencias sociales es muy bajo, dado el reducido niimero de alumnos 
que presentan sus tesis de maestria. Sobre la base de la informacion de 
aquellas maestrias en ciencias sociales cuyos datos sobre este punto 
eran confiables, hemos construido un indicador de desempeno, 
calculando el total de tesis presentadas hasta el momento de la 
acreditacion (correspondiente a fines de 1 998) por cada 1 00 
ingresantes, contabilizando como tales solo a aquellos que ingresaron 
con anterioridad a 1994 o 1993, si la maestria tenia una duracion de 36 
meses. Sobre 24 casos, un tercio de los programas tenia menos de 9 
egresados por cada 100 ingresantes y mas de la mitad, menos de 30 
egresados por cada 100 ingresantes (Garcia de Fanelli, 2000). 

En terminos de la calidad de las maestrias, se observa que las ciencias 

sociales alcanzan niveles de acreditacion y de categorizacion' * 
inferiores al promedio de las ciencias basicas. Por su parte, dentro de 
las ciencias sociales, las maestrias en economia presentan mejores 
estandares de calidad que las otras disciplinas de este campo de 
estudio (ver Cuadro N®4). 

Cuadro N" 4. Republica Argentina. Recomendaciones de los comites 
de pares de la CONEAU sobre los posgrados en ciencias sociales, 
humanas, economicas, basicas y aplicadas en 1999. (En porcentaje) 

( Fuente: CONEAU, 1999) 





Acreditados 


Areas de 
Estudio 


Sin 

acred itar 


No 

evaluables* 


Acred itados 


Categoria 

“A” 


Categoria 

“B” 


Categoria 

“C” 


No 

solicitd 

categoria 


Proyect 

nuevos 


Ciencias 
Basicas y 
Aplicadas 


7% 


0% 


93% 


38% 


40% 


21% 


0% 


1% 


Ciencias 
Sociales y 


34% 


1% 


65% 


8% 


35% 


39% 


6% 


12% 
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Humanas 


















Ciencias 

Economicas 


44% 


3% 


53% 


3% 


27% 


33% 


34% 


3% 



Por no estar la informacion completa 

Frente a estos resultados, caben al menos dos hipotesis. En primer lugar, es posible que los 

En segundo lugar, es factible que mas alia de los problemas del instrumento de evaluacion, 

4. Reflexiones finales: Oportunidades y limitaciones en el desarrollo de los 

La escasez de recursos y la ausencia de una clara orientacion en los planes de estudio, a lo 

Frente a un mercado de trabajo academico estrecho en cantidad y calidad de los puestos di 

Esta tension entre lo academico y lo profesional se expresa en programas de estudio que ni 

C^ne jicqblema para llevar adelante un programa academico avanzado en el nivel de la mae 
tienen los docentes en el nivel intemacional favorecen un intercambio mas dinamico y flui 

En el piano de la formacion academica en ciencias sociales, pareceria entonces importante 

Asi como parece necesaria una mejor articulacion entre las maestrias academicas con el do 

Si bien la consolidacion de posgrados de corte academico es necesaria para mejorar la call 

Esta modalidad de producir conocimiento se comienza a vislumbrar en algunos de las mae 

Tambien las maestrias profesionales e interdisciplinarias ban logrado cierto grado de “poro 

Al igual que en el caso de los programas de corte academico, una de las principals defici 

A diferencia de los posgrados de corte academico, los posgrados orientados a resolver pro 

Finalmente, en la consolidacion de estos programas es crucial que los controles de calidad 

La alta descentralizacion que prevalece en la gestion de estos programas de posgrado y su 

Notas 

* El CEDES es una asociacion civil sin fines de lucro de caracter independiente, fundada e 
^ Esta investigacion forma parte de un proyecto comparativo mas amplio realizado entre el 
^ Sobre la base de la frase del Evangelic segiin San Mateo que dice: “Pues al que tenga se 
^ Elios son, en primer lugar, una planta docente compuesta por profesores con alta experie 
^ La actual tendencia al aumento de una administracion institucional mucho mas fuerte y a 
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^ Siendo que en los posgrados de ciencias sociales, el niimero de alumnos del sector privad 
^ En las universidades piiblicas argentinas, solo el 13 por ciento de sus docentes tienen ded 
^ Fue un programa (en vigencia entre 1995 y el 2000) que asignaba fondos competitivos p 
^ Mientras que los estudios de grado en las universidades piiblicas son gratuitos, los estudi 
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Committing to Class-Size Reduction and 
Finding the Resources to Implement It: 

A Case Study of Resource Reallocation 

Allan Odden 
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Abstract 

This article discusses how a medium-sized school district 
in Wisconsin was able to reallocate resources to reduce 
class sizes in grades K-5 without spending more money or 
increasing its tax rate. Previous research on resource 
reallocation found that the bulk of reallocated resources 
were those supporting categorical program services. This 
district was able to use a different strategy. As a growing 
district, its marginal costs of adding an extra class of 
students were much less than its average expenditures per 
pupil, which were reimbursed by the state school finance 
formula. As the district grew, therefore, it acquired excess 
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revenues. Also, by implementing full-day kindergarten, 
the district acquired more excess revenues because this 
added (0.5 pupil) X (Number of kindergartners) to its 
current enrollment, and the cost of educating these 
students was less than the amount they received from the 
state funding formula. It then used these revenues to 
reduce class sizes to between 1 5 and 20 in all 
Kindergarten through grade 3 classrooms and to between 
15 and 22 for grades 4-5. 



Introduction 

Like most districts, the Kenosha Public School District in 
Southeastern Wisconsin faced a number of different educational and 
fiscal challenges as the new millennium approached. A medium-sized 
urban district in the Midwest, Kenosha served a population of 
approximately 20,000 students in the 2000-01 school year. 
Demographically, its student population was 77 percent white with 1 1 
percent Hispanic, 9 percent African American, and 2 percent other. 
Hispanics represented the largest and fastest growing minority group. 
About 30 percent of Kenosha's students lived in families with incomes 
below the poverty level, as indicated by eligibility for the federal free 
and reduced-price lunch program. Over the past decade, the percentage 
of students from low-income backgrounds continued to grow. 

But, rather than being evenly distributed across all schools, 
many of the district's low-income, minority and limited-English- 
speaking students were concentrated in a small number of its schools. 
Several years ago, it became obvious that the growing concentration of 
educational challenges that usually accompany these demographic 
characteristics was making it more difficult for students in these 
schools to achieve the high standards set for them by the state and the 
district. Although a growing student population, combined with a 
generous state education aid program, helped ensure budget stability, 
the district nevertheless struggled more and more to find the fiscal 
resources required to deploy programs successful in boosting the 
performance of its students, particularly its low income and minority 
students. 

In an effort to meet the achievement goals for all students over 
the 1990s, the district implemented a number of education reform 
initiatives. A new out-of-state superintendent who arrived in 1996 
stimulated these reforms, plus, as will be shown, a number of related 
management and fiscal changes. At the district level, the school 
system shifted from a junior high to a middle school approach to ease 
the transition into high school; opened three new charter schools to 
help foster innovation and new educational approaches; and most 
recently, implemented a full-day kindergarten program, in part to help 
low-income children learn to read in the early elementary grades. At 
the site level, many schools began to adopt new, structured literacy 
programs as well as comprehensive school reform designs, such as 
Direct Instruction, Success For All, the Literacy Collaborative, Marva 
Collins and Accelerated Schools. 

Although these reforms helped the district and its schools make 
progress toward their student achievement goals, teachers as well as 
district and school leaders wanted to see even greater progress. One 
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additional reform that teachers and administrators alike believed would 
provide a greater boost in student achievement was class-size 
reduction. 

Indeed, class size reduction, particularly in the early grades, was 
an education reform sweeping the country during this time. Odden and 
Picus (2000) found that 19 states and scores of districts enacted 
various versions of this education policy in the 1990s, believing that 
despite the high cost, it was a policy that research showed could 
dramatically raise student performance (Grissmer, 1999). 

Further, Wisconsin had adopted a targeted class size reduction 
program in 1995 as part of a statewide approach to improve student 
performance, particularly the performance of its low-income and 
minority students. Called the Student Achievement Guarantee in 
Education or SAGE program, it was initially limited to schools that 
had 50 percent or more of its students eligible for free and reduced 
price lunch, i.e., from a family with a poverty level income. In the 
1996-97 school year, this included 30 elementary schools in 21 
districts, including one school in Kenosha. For the 2000-01 school 
year, the program expanded its eligibility and had a dramatic increase 
in funding; in 2001, it helped hundreds of schools afford smaller 
classes around Wisconsin. 

Although there was strong support for a district-wide class-size- 
reduction program, there was still the issue of how to finance it. The 
expansion of the SAGE program provided one source. But two 
additional factors helped the district frame a strategy for how they 
could come up with the money to fond such a high-cost education 
reform. First, a 1994 legislative change in the federal Title I program 
allowed schools with at least 50 percent of its students in poverty to 
use their Title I funds for school-wide programs, rather than programs 
targeted to just its low-income students. The school-wide programs 
selected had to be research-based, and class-size reduction was a 
strategy that qualified. Second, prior to the opening of the 1998-99 
school year, district administrators heard a presentation on resource 
reallocation in education; one of the major ideas discussed was school 
use of Title I funds for schoolwide programs, including both class-size 
reduction and comprehensive school design models (see, for example, 
Odden & Archibald, 2000). 

The notion of resource reallocation for “whole school reform” 
triggered thoughts about additional programmatic and fiscal changes 
among many district and school leaders. The general notion was that 
the district could redesign schools from “the ground up” and use all of 
a site's resources to finance a new, research- based educational 
strategy, which could include small classes in at least Kindergarten 
through grade three. Further, a few elementary schools had access to 
other funds that could help reduce class sizes. In addition to Title I and 
the possibility of SAGE money, some elementary schools qualified for 
a small state program, called P5, that provided extra resources to 
schools with high poverty concentrations (schools could either receive 
P5 or SAGE money, but not both). In addition, the district knew that at 
the federal level. Title VI of the reauthorized Elementary and 
Secondary Education Act (ESEA) would provide an additional pot of 
money to help fond class-size reduction. 

A combination of the above ideas convinced the school board in 
July 1998 that it could begin immediately to reduce class size in a 
selected number of its high-poverty elementary schools just by 
reallocating Title I (mainly to school-wide programs) and P-5, and 



httn7/enfifl asii edii/enaa/vQn'lO html 



123 



10/29/01 




EPAA Vol. 9 No. 30 Odden and Archibald: Finding Resources for Class-size Reduction 

using SAGE and Title VI funds where possible. They did so, and in 
the first year they saw a positive impact on student academic 
performance as measured by state and district tests. 

In light of the positive results at these elementary schools, 
as well as even firmer belief in research supporting smaller classes 
(Finn & Achilles, 1999; Molnar, Smith, Zahorik, Palmer, Halbach & 
Ehrle, 1999, see also http://www.uwm.edu/Dept/CERAI), the district 
decided to find the money to reduce class sizes for Kindergarten 
through grade three in all elementary schools the following school 
year. Whether this was the reform that would best help all students is 
beyond the scope of this study. Instead, this case study tells the story 
of how the district was able to fund this ambitious class-size reduction 
initiative. 

Finding the Money 

Previous research on resource reallocation (Achilles, 1 999; 
Odden & Archibald, 2000) showed that categorical program dollars — 
Title I, special education, programs for limited- English proficient 
students, etc. — comprised the bulk of funds that were reallocated, 
whether the goal was class- size reduction or implementing a specific 
whole school design, such as Roots and Wings/Success for All. 
Kenosha initially used this strategy as well, as it was primarily federal 
Title I, Title VI, and state SAGE and P5 money that allowed the 
highest poverty schools to begin to reduce class sizes. Studies have 
shown that reduced class sizes are especially beneficial for low-income 
and minority students (Finn and Achilles, 1999). 

Believing these benefits would extend to all students, the district 
decided to reduce class sizes in all elementary schools, including those 
that were not eligible for categorical program funds, but to do so it 
needed a new, larger source of funding. This challenge sent the district 
back to analyze the workings of its overall budget as well as the state 
of Wisconsin's school finance equalization formula. These analyses 
also prompted the district to consider implementing a full-day 
kindergarten program. Although they had been considering this step 
for a long time, this decision, combined with the reality of rising 
student enrollments, helped the district generate the resources that 
would enable it to fund a larger class-size reduction strategy. 

Funding for the larger class-size reduction initiative came from 
two sources: rising enrollment and the implementation of full-day 
kindergarten. In the next few paragraphs, these funding sources will be 
addressed in turn. First, Kenosha was able to generate resources from 
rising enrollments in the following way: In Wisconsin, districts receive 
aid on the basis of a three tiered, guaranteed tax base, school finance 
formula. The state aid formula functions so that in any one year, for 
each additional student Kenosha gains (or loses), the district receives 
(or loses) approximately $7000. Since the district has experienced 
rising enrollments in the past few years, its budget has constantly 
grown. And because each new student costs less than $7000 to 
educate, the marginal cost of educating each new student is less than 
the average per pupil expenditure of the district. To illustrate this 
point, assume the district enrolls 25 new students. The district’s budget 
would rise by $175,000 (25 x $7000). But the district would have to 
hire only one additional teacher for a new class of 25 kids. This would 
require one teacher at an average cost of $50,000, plus the cost of 20 
percent of another teacher for the first teacher's planning and 
preparation time, and some additional costs for materials, supplies and 
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operations and maintenance. Assume these costs totaled $75,000. That 
would leave the district with excess revenues of $100,000, which 
would be sufficient for the district to hire close to two additional 
teachers, at an average cost of $50,000 each. The marginal costs for 
the additional 25 students were only $3000 per student ($75,000 
divided by 25 students), while the average additional dollars were 
$7000 a child, for a possible difference between average and marginal 
costs of about $4000 a child. 

To be sure, the above is a simplified analysis. For the numbers 
to work exactly, all students would need to be in the same grade level 
and in the same school. But since the district enrollment grew by 
approximately 500 students between 1999-2000 and 2000-01, a rise of 
25 per grade was possible, though maybe not in each school. For these 
reasons, it may be true that the difference between average and 
marginal costs for each additional student is not precisely $4000. Even 
so, because of the high level of the school finance equalization formula 
and the phenomenon of rising enrollments, the district was in the 
enviable position of generating substantial excess revenues for each 
new student in the district. This situation was further strengthened by 
the fact that the district had some excess classroom space, so it did not 
need to build new classrooms because of enrollment growth or because 
of class size reductions. How the district found the space for the extra 
classes will be discussed later in this case study. 

Another complication for the above example is that Wisconsin's 
education aid formula does not use the current student count in 
determining each district's aid, but uses the average of the number of 
students from the past three years. Initially, this was implemented to 
limit state aid losses for declining enrollment districts, but for growing 
enrollment districts it limits the increase in revenues as well. Thus, for 
Kenosha to enjoy the full measure of the excess of average over 
marginal cost increases, it needed a full three years. If this were the 
only source of new funding, the district would have had to phase in its 
class size reduction strategy over several years. 

However, as previously mentioned, the district found another 
way to raise revenue for the small class-size policy that further aided 
all its students, but particularly its students from low-income 
backgrounds, which was the implementation of full-day kindergarten. 
In the mid-1990s, the district provided only a half-day kindergarten 
program. But it knew that full day kindergarten was desired by many 
families and also was a research-based strategy to help students from 
low-income backgrounds learn reading and writing in the first three 
years of elementary school (Slavin and Madden, 1 996). Therefore, the 
district decided to change from a half-day to a full-day kindergarten 
program. This move generated more funds for class-size reduction for 
the same reason identified with new enrollments: having 
kindergartners for an extra half day gave the district additional 
revenues because of excess average over marginal costs. Because the 
state allowed the district to count the child as a 1 .0 student for a fiill- 
day program as compared to 0.5 for a half-day program, the shift to a 
full day program essentially allowed the district to “increase” 
enrollments on its own. For every two students who shifted from a 
half- to a full-day kindergarten program, the district was able to add 
1 .0 (2 X 0.5) students to its pupil count. It received the full $7000 for 
this “extra” student, but again its marginal costs were much lower than 
this $7000 figure. 

Another example helps illustrate this point. When a school has 
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20 students for morning kindergarten and 20 students for afternoon 
kindergarten, the school receives 40 x 0.5 x $7000, or $140,000. 
Assuming that the same teacher is working in the morning and the 
afternoon, the school will spend approximately $50,000 of that amount 
on the teacher's salary and benefits, plus planning and preparation 
time, materials, etc. which might total $65,000 - 70,000, leaving an 
upper-bound estimate of $70,000 ($140,000 - 70,000) in excess 
revenues. 

When those 40 students change to spending a full day at school, 
the school receives $7000 for each student, or a total of $280,000. The 
school will now need to hire two teachers, who will each have a class 
of 20 students, which will cost about $140,000 ($100,000 for salary 
and benefits plus additional money for planning time and materials). 
This leaves $140,000 ($280,000-140,000) for other purposes, 
compared to the $70,000 that was left when the 40 students only 
attended for half a day. Because Kenosha had most of the necessary 
classroom space, the district primarily needed only to build in the cost 
of operating those rooms all day to cover the expenses of essentially 
doubling the number of kindergarten classes by moving to full-day 
kindergarten. As is true with the growing number of students enrolled, 
the marginal cost of educating kindergartners for a full day was less 
than the average amount that they received via the state equalization 
program, resulting in a net fiscal gain for the district. Again, because 
of the three-year average pupil count, these excess revenue numbers 
took three years to be folly realized. 

So the district was able to take advantage of two rising 
enrollment phenomena to produce the revenues to help fond a district- 
wide class-size reduction program — first, excess revenues from 
naturally occurring enrollment increases, and second, excess revenues 
from shifting from a half-day to a full-day kindergarten program. 

When combined with the initial categorical dollars that were used to 
reduce class sizes first in the highest poverty-concentration schools, 
Kenosha was able to fond dramatic class size reductions without 
increasing taxes, eliminating other programs, or increasing its average 
expenditure per pupil. 

Implementing the Class Size Reduction Policy 

The two “new” revenue sources discussed in the previous 
section made it possible for the district to help fund reduced class sizes 
at all of its elementary schools, but implementing this initiative was 
complicated. First, because the excess revenues from shifting to a foil- 
day kindergarten program would be phased-in over a three-year 
period, the superintendent initially advised the district to slowly 
implement the class-size reduction policy, making sure that the 
resources were there before putting the plan into action. However, the 
board voted to move ahead immediately because it felt that when their 
strategy for finding the money to do so became public information, 
there could be many other claims for its use — including other 
programs as well as teacher salary increases. Although class-size 
reduction was very popular with teachers, the policy would be to 
reduce class size for just kindergarten through grade 5, so teachers of 
other grades could very well have had different ideas about how to 
spend those dollars. Though the superintendent initially was skeptical 
about moving so fast, he eventually came around because he saw the 
wisdom of quickly implementing a popular policy — class size 
reduction — and avoiding a lengthy debate over alternative uses of the 
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“found” dollars. 

In order to avoid the potential delay, district and board leaders 
acted quickly. In July of 1999, the Kenosha school board accepted the 
district proposal to fund 22 new teaching positions to reduce class 
sizes for the following school year. This meant that the district had just 
over a month to hire all of these teachers plus the normal new hires for 
the year. It managed to accomplish this task, hiring 37 new teachers 
(22 for class-size reduction) in the one month before school began in 
September. This was one of the difficulties with trying to implement 
this initiative so quickly. 

The 22 new teachers were awarded centrally — schools had to 
request these extra teachers from the office of the superintendent and 
explain how they would be used to reduce class sizes and what space 
they would use. Because the district had staffed each elementary 
school on a 24 students to 1 teacher basis, each school was required to 
use the new teachers for class sizes below that level. Some schools had 
the extra space, especially the high-poverty elementary schools that 
had declining enrollment. Other schools had to give up their art, 
music, or multipurpose rooms, which some schools were reluctant to 
do. Some sacrifices had to be made to implement this policy, but the 
district had made the decision to reduce class size, and to do so they 
needed classroom space. These initial new teacher positions helped 
bring down class sizes somewhat, but district leaders had begun to 
make long-term plans for formally reducing class sizes even further, to 
specific target levels. 

Over the course of the 1999-2000 school year, district leaders 
had more time to plan the further reduction of elementary class sizes in 
the Kenosha School District. This time, they decided to reduce the 
staffing ratios for grades K-5 rather than centrally award “extra” 
teachers to the sites, which had produced a variety of different but 
lower class sizes. This district-wide policy change made the class-size 
reduction policy explicit for all elementary schools. 

As previously mentioned, the ratio that had been in place was 
one teacher for every 24 students in grades K-5. In the 2000-2001 
school year, with the new class-size reduction policy in place, the 
ratios became 20 to 1 for K-3 and 23 to 1 for grades 4-5. These ratios 
were set in part according to how many extra elassrooms were 
available at the schools; although district leaders would have liked to 
have even lower class sizes, this was the level at which they felt 
schools had the appropriate extra space to accommodate the additional 
classrooms. The class size maximums also changed. Previously, the 
maximums were 27 for kindergarten through grade 3, and 29 for 
grades 4-6; under the new allocation formula, maximums were set at 
22 for kindergarten through grade 3, and 25 for grades 4-5. 

Further, in the past, when a class exceeded its maximum, an 
instructional aide was placed in the classroom to provide assistance. 
More specifically, if the class size was above 26 but not above 30, a 
half-time instructional aide was provided; a full-time aide was 
provided if the actual class size was over 30. But under the new, 2000- 
2001 class-size- reduction policy and class-size maximums, most of 
the resources for instructional aides were reallocated for the additional 
classroom teachers that were necessary to keep class sizes lower. The 
Assistant Superintendent of Instruction stressed that the district had 
adopted an overall value that, whenever possible, resources would be 
used to employ fully trained and licensed teachers rather than 
instructional aides. This decision was bolstered by the fact that the 
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same research that documented the effectiveness of small classes also 
showed that adding an instructional aide to a larger class provided no 
effect on student performance (Achilles, 1999). Indeed, many of the 
elementary schools that received Title 1 funds eliminated instructional 
aides and used their Title I funds to hire teachers instead. In Kenosha, 
it takes the resources for about two full-time aides for a school to hire 
a licensed teacher. 

For the low-performing schools in Kenosha, class size targets 
were set even lower than at the other elementary schools. Kenosha 
defined low-performing by student performance on the Wisconsin 
Knowledge and Concepts Examination, a statewide testing system 
using the Terra Nova commercial test. All schools with an average 
WSAS score below the state average were categorized as “low- 
performing.” Even before the district changed its staffing allocation 
formula, these schools, because they had the highest poverty 
concentrations, had access to all of the previously mentioned 
categorical program dollars that could be used to help fund class-size 
reduction: Title I, Title VI, P-5 and SAGE. 

The district class-size goal for low-performing, high- poverty 
schools was set at 1 5, with a maximum of 1 8. This maximum was set 
because schools receiving Title VI funds, the federal class-size 
reduction program, had to reduce class sizes to 1 8 or lower. Thus, the 
district class-size goal for the low performing schools is lower than the 
federal requirement. The extra funding from SAGE, Title I, Title VI 
and P-5 that these schools received was used to enable the district to 
reduce class sizes to 1 5 in many of its low-performing schools. 
However, because there are more low-performing elementary schools 
in the Kenosha School District than resources to reduce class sizes to 
those target levels, the district focused on reducing class size to these 
low levels at the lowest performing schools first. 

As of late 2000, the district had made significant progress in 
reaching its lower class size goals. Of the 24 elementary schools in the 
district, one had class sizes of 1 5 or lower, eight niore had class sizes 
of 1 8 or lower, 1 1 had class sizes of 20 or lower, and four had class 
sizes of 22 or lower. Thus, all elementary school class sizes were 
below the maximum of 22 for grades K-3, and well below the 
maximum of 25 for grades 4-5. Further, while only one high- poverty 
school had an average class size of 1 5 or lower, all elementary schools 
at which the majority of students are low-income had class sizes of 20 
or lower. 

For many elementary schools, the combination of the new, 
lower staffing ratio and the reallocation of categorical funds has meant 
a dramatic reduction in class size and a feeling of optimism about the 
subsequent effect on student achievement. For example, at Wilson 
Elementary School, where 86 percent of the students qualify for free or 
reduced-price lunch, class sizes have been reduced from an average of 
21 .9 in 1998-99 to an average of 16.4 in 2000-01 . In order to reduce 
class sizes that far, the school needed a total of nine more classroom 
teachers in 2000-01 thiin it had in 1998-99. The reduction in class size 
was possible because of the change in the staffing formula and the 
allocation of categorical funds for class-size reduction. Enrollment 
growth does not explain any of the new teachers for 2000-200 1 ; 

Wilson had only 10 more students in that year than in 1998-99. The 
school received three additional teachers for 2000-01 because of the 
new staffing allocation formula. The other six teachers were paid for 
with categorical funds: one with other district funds for class- size 
reduction, two with federal Title VI funds, 2.5 with SAGE funds, and 
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0.5 from Title I. The district flinds for class-size reduction were the 
result of a board decision that 30 new teachers would be hired for the 
2000-200 1 school year; Wilson got one of those teachers. It is also 
interesting to note that this had formerly been a P-5 school, but they 
traded in that status in order to qualify for SAGE since they were able 
to hire more teachers with those funds. 

By concentrating many of its resources on class-size reduction, 
Wilson was able to lower class sizes to an average of 16 students. This 
is noteworthy because it is within the range of 13-17 identified by the 
Tennessee STAR study as being especially effective at raising student 
achievement levels and producing other favorable outcomes (Finn & 
Achilles, 1999; Grissmer, 1999). 

Class sizes have also been significantly reduced at Vernon 
Elementary School, where 36 percent of students qualify for the free 
and reduced-price lunch program. Although this is a significantly 
lower percentage than at Wilson, Vernon still receives substantial 
categorical funding. As was true at Wilson, it is this funding in 
combination with the district's lower staffing allocation formula that 
has allowed Vernon to reduce class sizes from an average of 27 in 
1998-99 to an average of 18 in 2000-01. Enrollment has stayed 
virtually the same over this time period; 468 students attended Vernon 
in 2000-01 as compared to 472 in 1998-99. Therefore, none of the 10 
additional teachers on staff at Vernon in 2000-01 can be explained by 
enrollment growth. Instead, four of the teachers were a result of the 
change in the staffing ratio from 24 to 1 to 20 to 1 ; one was paid for 
using the additional district funds for class- size reduction; one was 
paid for with federal Title VI funds; three were paid for with Title I; 
and one with P-5 funds. By concentrating its funding sources on class- 
size reduction, Vernon was able to reduce class sizes by a remarkable 
nine students per classroom. 

Initiatives Supporting Class Size Reduction 

In addition to getting class sizes down to desired levels, part of 
the implementation process is to provide teachers with professional 
development that will help them teach their small classes more 
effectively. Leaders in Kenosha knew that research has shown that 
teachers can help produce additional achievement gains when 
professional development in small-class instruction is provided 
(McRobbie, Finn & Harman, 1998; Bearl, 1998; U.S. Department of 
Education, 1998). The Assistant Superintendent of Instruction stressed 
the importance of this aspect of implementation, and has continued to 
seek School Board support for substantial investments in professional 
development in order to help realize the potential of better instruction 
in the smaller classes. Title VI, the federal funding source for class- 
size reduction, allowed districts to use 15 percent of its funding for 
professional development in 1999-2000; although district 
administrators thought this would be a good idea, the board voted to 
use all the funds to pay for teachers in the first year. However, for the 
2000-2001 school year, districts were allowed to use up to 25 percent 
of their Title VI funds for professional development, and this time the 
board agreed, in part because the funding had increased and they were 
able to hire almost as many teachers and have money leftover for 
professional development. The district was able to allocate $137,000, 
or approximately 25 percent of the federal money they receive for 
comprehensive school reform, toward professional development 
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programs designed to improve instructional practices in small 
classrooms. This money made it possible for 700 of the district's 1600 
teachers to participate in such professional development activities as 
Everyday Mathematics training. 

Further, the district has continued to encourage all schools to 
adopt some type of “whole school” educational strategy to accompany 
both the above professional development and small class sizes. For 
example, for the 2000-2001 school year, the district provided 
$299,000 in grants to schools without P5 or Title I funds to help them 
afford various school reforms, including comprehensive school reform 
models. This district's commitment to raising student achievement 
scores is apparent through the creative use of all of their funds to make 
changes that they believe will boost student achievement. 

Monitoring Results 

Although the superintendent and other district leaders have 
made class-size reduction a priority in Kenosha elementary schools, 
they recognize that it is just one reform to be used in conjunction with 
other reforms for a common purpose: to boost student achievement. 

For that reason, student achievement scores are carefully measured and 
reported in order to track the success of these reforms. In 2000-01, 
students at Kenosha elementary schools took the Iowa Test of Basic 
Skills (ITBS) in grades 2, 3, and 5; the Wisconsin 3rd grade reading 
test, and the Wisconsin IQiowledge and Concepts Examination 
(WKCE) in 4th grade. The district placed the most emphasis on the 
3rd grade reading test and WKCE. Each school had benchmarks that 
they were working toward, which were set in terms of the percent of 
students at proficiency level and the percent advanced. Awards were 
given to schools that increased these percentages by five percent 
annually. 

Student achievement growth has been substantial at many 
Kenosha Elementary schools, including Wilson, one of the schools 
used as an example in the last section. In addition to reduced class 
sizes, Wilson has used the principles of the Marva Collins school 
design for the past few years, and has more recently adopted Direct 
Instruction for reading, and Core Knowledge as a curriculum guide. In 
the 1997-98 school year, only 17 percent of Wilson students were 
considered proficient on the third grade reading test. In 1999-2000, 51 
percent were at or above proficiency — a dramatic improvement of 34 
percentage points. 

Columbus Elementary School, a school with 58 percent of 
students who qualified for free or reduced-price lunch in 2000-01, has 
also made great strides. Funded by their categorical dollars, Columbus 
has had reduced class sizes for three years. The school chose class-size 
reduction rather than a comprehensive school reform model, and is 
now listed in the high achievement category for the WKCE in Math. 
The state average score for percent proficient in math is 52, and 68 
percent of Columbus students scored at or above proficiency (Barth, 
Haycock, Jackson, Mora, Ruiz, Robinson, & Wilkins, 1999). This is 
the best indication so far that the policy to reduce class sizes in 
Kenosha will boost student achievement in all schools, as it has at 
Columbus Elementary. In addition to Columbus and Wilson, student 
achievement has risen at many other Kenosha schools, in part due to 
initiatives like class-size reduction and comprehensive school reform 
models. 
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Conclusion 

Leaders in the Kenosha School District in Wisconsin managed 
to significantly reduce class size in the majority of their elementary 
schools — by creative resource reallocation and deployment of all the 
revenues made possible by student demographic characteristics and the 
state's school finance system. In the 2000-01 school year, more than 
one-third of Kenosha's elementary schools had class sizes of 18 or 
lower, and all 24 schools had class sizes at or below 22. Although 
individual schools in this district had begun to reduce class size 
school-by-school by reallocating Title I and P-5 funds and using Title 
VI and SAGE, the district-wide change to a lower elementary school 
staffing allocation formula made sure that class sizes were reduced at 
every elementary school, and to lower levels than the schools could 
have afforded with just categorical dollar reallocation. 

The major revenue source for this expensive policy was excess 
revenues derived from the combination of growing enrollment and the 
shift from a half-day to a full-day kindergarten. For every new student, 
the marginal cost of educating that student was approximately $3000 
but the district received an extra $7000 via the state school finance 
formula, or an excess of average over marginal costs of $4000 per 
child. The total combined district enrollment growth from these two 
phenomena — natural growth and kindergarten expansion — was about 
500 students a year. This produced excess revenues of nearly 
' $2,000,000 (500 students times $4000/student), which was sufficient 
to hire 40 additional teachers at an individual cost of $50,000 in salary 
and benefits. This quite ingenious way to fund smaller class sizes, 
combined with additional dollars from selected categorical 
programs — federal Title I and Title VI, and state SAGE and P5 — 
allowed for even lower classes in the highest poverty, lowest 
performing schools, reaching the level of 1 8 or lower in nine of the 
district's 24 elementary schools. District leaders hope that the positive 
results from Columbus Elementary School, which has had lower class 
sizes in place for three years, will be replicated in elementary schools 
district-wide. 

This case shows how important it is for district leaders who 
want to make changes using reallocated dollars to have full knowledge 
of the district budget and how that budget is derived. In Kenosha, 
district leaders decided to reallocate categorical dollars to class-size 
reduction, but they needed an additional source of funding to reduce 
class sizes to target levels. They were able to find that additional 
funding source because they understood the principle that “new” 
students, whether from natural enrollment grovs1;h or the shift from 
half-day to full-day kindergarten, could produce “new” dollars because 
of the excess of average over marginal costs. These changes enabled 
the district to provide full-day kindergarten and reduce class sizes in 
all elementary schools, initiatives that research shows are particularly 
powerful in helping students from low-income backgrounds learn to 
read and do mathematics in the early elementary grades (McRobbie, 
Finn, & Harman, 1998; Slavin and Madden, 1996; Slavin, Karweit and 
Madden, 1989). 

Note 

This article was prepared for the Consortium for Policy Research in 
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Abstract 

Local actors' perceptions of curricular and management 
changes in two private schools and one neighboring 
public secondary school in the city of Buenos Aires, 
Argentina, are analyzed. An exploration was conducted of 
how, within an ideologically and politically pro-reform 
context and a widespread acceptance of the "private 
school advantage," principals, teachers, and students in 
these schools evaluated the changes (or lack of them) in 
management, teaching, and curriculum orientations of the 
secondary education sector. 
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Introduction 

In the debate over implementing “choice and free-market” mechanisms to correct the deficiencies of pu 
In some countries, and in spite of Adam Smith’s recommendations the once quixotic aspiration that ev 
The coexistence of public and private schools as compatible providers of education, each targeting a di 
By conducting an exploratory qualitative study of public and private high schools, this project aimed at 
The empirical data are drawn from three schools and, consequently, can not provide broad-based concl 
In the attempt to map and locate the changes happening simultaneously at the local and global level usi 
I. The “Private School Advantage” 

The notion that the provision of educational services should be regulated by the market or in a competi 
Perhaps no other work has contributed more to that debate in the USA and abroad than Politics, Marke 
Chub and Moe’s proposed solution is to incorporate market-like incentives and discipline in education 
The central argument ... is that the success of any human organization depends on the unificatio 
The assumption is that a quasi-market will wisely solve contemporary educational problems since pare 
There are, however, numerous objections to the implementation of market-like proposals and to increas 
The emphasis on equality means that the focus in education is on the bottom of the performance 
The pro-market choice model advanced by Chubb, Moe, Coleman and others relies heavily on what Jo 

• Better performances on standardized tests; 

• Higher graduation rates; 

• More rigorous academic environments; 

• Safer schools; 

• More opportunities for their parents to participate; 

• More access to morally uplifting surroundings (very often in association with religious based tea 

• More access to highly motivated teachers and administrators (motivation based on efficient and 1 

The works of Chub and Moe, Viteritti and others indicate that in general, there is a strong widely held 
Because private schools can select (and are selected by) their students, analysts have not been abl 
Rothstein, Carnoy and Benveniste also noted that solving the debate about the reasons for the supposed 
My reading of the theory and empirics of literatures concerning educational production functions 
Privatization, market-like options, choice programs, and vouchers are at the forefront of the education 
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Without doubt, these actions , which tend to the partial or total marketization of the educational sector c 
These reform proposals are made not only by the staff of think tanks, the media, and government offici 
It is at this crossroads of global neo-liberal reform proposals, generalized beliefs about the distinct perc 

II. Argentina’s Search for “Modernity” 

Argentina is a country of 35 million people, and it has one of the lowest population growth rates (1 .6% 

The old optimistic picture was eroded by a profound stagnation in production that led to a steep 

Idnfortunately, Schvartzer is not alone in his description of the country’s political, social and economic 
social discussion. 

After World War II, as in most countries in Latin America, Argentina’s society debated and in many ca 
(I) social modernization (economic growth with the incorporation into the world market, urbaniz 
Not surprisingly, during the second half of the 20th century, the prevailing models of development and 
In Latin America, the modernization approach required the identification of sectors or areas which wer 
First, the State itself has to become modem and encourage appropriate civic behaviors through rational 
Between the 1930s and 1960s, the Argentinean State played an important role in such a selective proce 
Nevertheless, this modernization model did not last long in Latin America and its weaknesses became 
The financial crisis of the 1980s, the lack of social and political effectiveness of many of the dictatorial 
During the 1980s, the region witnessed a cycle of inflation, hyperinflation, and recession of a severity 
Doubtless, the early 1980s were crucial in Argentina. The financial crisis, the Malvinas (Falklands) Wa 
Nevertheless, this model of stabilization and adjustment which emerged after the so- called “Washingt 
By the end of the 1990s, the policies of economic and financial stabilization and adjustment had also in 
During President Carlos Menem’s governments (1989-1999), Argentina experienced an accelerated pr 
Most of those changes were justified as the only possible solution to the economic problems of the cou 
In addition to these economic measures, it should be noted again that the problem of achieving the “rig 
The macro-economic indicators have changed and according to external observers they have improved 
Important transformations existed not only in the economy but also in the education sector. The “new 

III. Notes on Data Collection 
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The data for this research were gathered during two three-month periods and through school observatio 
The selection of these three schools was based on geographical proximity and similarities in student po 
It should be noted that — even though this limits even more the results of this research — none of the tw 
In addition to the activities carried out within the secondary schools, the researcher interviewed the dir 

IV. In Search of Educational Modernity 

An anecdote illustrates some of the pivotal points in which the search for “educational modernity” was 
May 1993: visit to the ministry of education for interviews. I have to wait for the “customary” 30 
Despite the content of this measure, the initial reaction of this researcher was: “This is a change f 
These notes contain some of the tensions developed in the process of searching for a new model of edu 
The first such initiative occurred in 1 992 when the federal government developed a process of decentra 
At first glance, through this transference the federal government, would transfer power by not controlli 
[H]as retained responsibility for four major areas: set of common contents, evaluation of outcom 
There is a general consensus among researchers that the decentralization process initiated in 1992 paral 
It is relevant to quote Hanson’s account of how the decentralization process took place: “The transfer o 
The second initiative was the sanction of the Federal Law of Education, the major legal education refor 
The structure of the academic system: Previously, elementary school was mandatory for 7 years and 
The curricular standards: The Law establishes the criteria for the development of the minimum requi 
Teachers’ Professional Development: A national network of teacher professional development activit 

Development and implementation of a national system of assessment: The national ministry of edu 
The economic-financial rationale and pro-private overtones of both laws were widely recognized in the 
First of all, it is possible to say that the private education sector has benefited from the implemen 
While debate of the law was taking place, the government was firmly committed to the implementation 
In all of these quality evaluations (Mendoza, SINEC National, and other smaller evaluations), the resul 
More importantly, as the data in Table 1, 2 and 3 illustrate (especially given the objectives of this resea 
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Source: Llach, J. J., Montoya S., and Roldan, F. Educacia Para Todos, Cordoba, IREAL, 1999. 



Source: Argentina Ministerio de Cultura y Educacion (1998) 



Source: Argentina Ministerio de Cultura y Educacion (1998) 

The initial outcry about the results was followed by a great media battle over whom to blame. Teacher 
In Argentina, the participation of the private sector in secondary education is quite important and enroll 
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Source: Argentina Ministerio de Cultura y Educacion (1998) 

Since 1970, enrollment in private secondary schools has stagnated, yet the number of schools has incre 

The Principals 

The interviews with the principals of the three schools revealed two main trends. The first trend refers t 
The first trend requires some background information. What distinguishes the case of private schools i 
Principal Public School PS: It is scandalous that the government spends money in (name of priva 
For both principals in the private schools, (thereafter Mountain-view and Lake-view) the public subsidi 
Private School Mountain- View principal: The subsidies are important, without them we cannot 
During the period 1995-1998 the school of principal of Mountain-view saw a small decrease in the nu 
Question: In your opinion, is the private sector trying to reach other social sectors, the lower mid 
Private Schools Consultant: I do not see any interest in reaching the lower middle class, at least i 
This consultant reflection pointed out to the sad reality of the absolute imperfection of Argentina’s edu 
In light of the opinion of the private schools’ consultant it is not surprising, then that the principal of M 
In the case of the principal of Lake-View school the subsidy was also seen as needed but with the addit 
Private School Lake-View principal: I would love to get ride of the subsidies, but we can’t. No s 
Question: What would be the advantage of not receiving subsidies? 

Lake-View principal: Some schools are trying that. The first advantage is that there is no more re 
Despite the difference of opinion about the role of the state subsidies, the principals of the three school 
Mountain-View principal: You know, in the private schools it is easier to find room for innovati 
When the first part of the fieldwork for this study was undertaken, the debate about education (or the cr 
Question: How do you evaluate the performance of private schools in general given the results of 
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Mountain-View principal: The important thing here is to recognize that our school is doing the w 
Question: How do you explain that on average the scores of students in private schools and publi 
Mountain-View principal: Well the problem is the test and its evaluators. The tests were prepare 
Similarly, the principal of the public secondary school expressed discomfort when asked about the nati 
Question: What do you think about the quality of education evaluations? 

Public School principal: I don’t like the use of these tests; it is all political... we need to ask why 
As these testimonies indicate, the principals in the private and public schools shared concerns about the 
The idea that private schools have more flexibility than public schools o change or adapt the academic 
The three principals were very concerned with security and discipline as a way of improving the condit 
It is very likely that the concern with discipline expressed by the principals was a response to a few vio 
The Teachers 

The interviews with teachers confirmed the trend about the “private school advantage” but also reveale 
Teresa P., a female teacher, has been teaching history in private and public schools for more than five y 
Question: Where do you feel better? At the public or at the private school? 

Teresa P.: Nowhere; I have the same problems, the place doesn’t matter, the same low salaries, t 
Question: Can you think about any difference? 

Teresa P.: In the bathroom [laughing] and that is serious! I mean the lack of investment that you 
The testimony of Teresa P. exemplified a common pattern expressed by several teachers during the cou 
A possible explanation of the “society first” trend can be found in the severity of the country’s financia 
There is a national trend of a prolonged decline in the real salaries of teachers from the mid- 1970 
In the late 1990s, teachers were earning a third less than an average worker in the formal economic syst 
Teachers were blamed for declines in student achievement and lack of consideration about students’ w 
It is in this context that the relationships between teachers and parents and teachers with administrators 
Returning to the central focus of this research, the comparison of the perceptions between secondary pu 
Carlos P. (Lake-View): I started my career full of energy! I felt that teaching was my mission, m 
Question: Anything else? 
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Carlos P.: Oh yeah! But it is not what you are thinking about! At Lake-View you have someone 
Question: What do you mean? 

Carlos P.: Students, not all of them, but some, sometimes mistake the fact that they are paying fo 
Question: Do you consider that as a challenge or that this kind of pressure makes you change yo 
T.2Prll: Oh no! At the public school you have something similar, teenagers are always challengi 
This teacher’s testimony captures several key questions for this research. Is it possible to frame public 
Finally, in the interviews with teachers it is difficult to asses any significant difference in terms of priv 
Autonomy is an issue that does not clearly distinguish public from private education. The freedo 
The Students 

The findings from focus groups and interviews with students from public and private schools revealed 
As in the case of the principals, a similar strong belief was reported about the private school advantage 
Sudent.l -Lake-View: my parents are always reminding me how much my school costs, and I tell 
A male student in his last year of secondary public school reflected about his experience in the followi 
Student.2-Public School: If you look around this school, you will see how depressing it is to be h 
When questioned about possible differences between students in private and public schools Student.2 i 
Student.2-Public School: If I have to judge from my friends that go to the private school, I think 
The comments of Student.2 are illustrative of a strong tendency among the students. They do not know 
The testimonies of Student 1 and Student2 are clear examples of the disappointment with schooling, wh 
Student.3-Public School: I support teachers demands for better salaries, no doubt that the govern 
Question: Do you think that your teachers are going to participate in the protests? 
Student.3-Public School: My teachers? No, . . . most of them are scared and old- fashioned. They 

The notion that teaching is a noble profession — but somehow the real teachers working at real schools 
Student.4-Lake-View: No, teachers here are teachers. 

Question: What do you mean by “teachers are teachers”? 

Student.4-Lake-View: . . . They don’t want to risk anything, they do everything the principal tells 
Student.4-Lake-View: Yes, she is totally right, there are days we spend as much time sleeping in 
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The previous testimonies are consistent in showing dissatisfaction with schooling and authorities in ge 
Student.S-Public School: It couldn’t be different, we share the same teachers! 
Sudent.6-Mountain-View: Yeah! Mr. Pepe teaches the same nonsense here [private school] and t 
Student.S-Public School: Besides, you know, we are experts! 

Question: What do you mean? 

Student.S-Public School: You know, cheating is very common; I think that when we finish scho 
To sum up this section, students from public and private schools consistently supported the notion of th 

V. Conclusions 

This research was conducted to assess the extent of curricular and management changes in two private 
It is important to restate that this study did not intend to provide generalizable conclusions valid for all 
After reviewing all the evidence gathered, the presumed capability of private schools in absorbing incr 
Besides supporting the “private school advantage” notion, both teachers and students in these three sch 
Contrary to what the “free-markef ’ literature predicts, in the case of these three secondary schools in B 
The findings of this study are also consistent with the results of Glass’s (1997) study about public and 
[T]he social, cultural, and economic backgrounds of the parents and the community in which the 
The three schools studied in Buenos Aires operated in a context in which their pedagogical and admini 
The importance and influence of the current debate in Great Britain and the USA about markets and ch 
In the current globalized capitalism, the pressures to transform the models of modernization, the role of 
Under these disciplinary constraints the contemporary Latin American state can be best characterized a 
Financial institutions such as the World Bank and the International Monetary Fund respond to the polic 
Established Western societies are no longer optimistic that the Welfare State can effectively assis 
As the current North American debate over vouchers and choice initiatives shows, the pro- market initi 
At this point, with the available evidence from this small research in Argentina, and the international e 
When the available information is assembled in the U.S. and abroad, the evidence suggests that “ 
The findings of this research support Camoy’s remarks. It appears evident, then, that in the case of Arg 
I would, however, recommend that further studies about this topic follow Adam Przeworsky cautions r 
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This is a threefold question: (1) What are the economic costs of such transformation? (2) Under what p 
We have no theory of structural transformation, and the empirical evidence is scanty. Market ori 
I would like to conclude that today, nine years after Pzreworsky’s contention, the empirical evidence is 
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Abstract 

In this article we discuss the effects of one urban school 
district's efforts to increase the autonomy and 
accountability of schools and teams of teachers through a 
standards-based reform known as team- based schooling. 
Team-based schooling is designed to devolve decision- 
making authority down to the school level by increasing 
teachers' autonomy to make decisions. Increased 
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accountability is enacted in the form of a state-level 
standards-based initiative. Based on our evaluation over a 
two-year period involving extensive fieldwork and 
quantitative analysis, we describe the ways that teachers, 
teams and school administrators responded to the 
implementation of team-based schooling. What are the 
effects of increasing school-level autonomy and 
accountability in the context of standards- based reform? 

Our analysis highlights several issues: the "lived reality" 
of teaming as it interacts with the existing culture within 
schools, the ways that teachers respond to the pressures 
created by increased internal and external accountability, 
and the effects of resource constraints on the effectiveness 
of implementation. We conclude by using our findings to 
consider more broadly the trade-off between increased 
autonomy and accountability on which standards-based 
reforms like team-based schooling are based. 

I. Introduction 

The standards movement was one of the key reform strategies 
developed in the 1 990's as states and districts sought ways to raise 
student achievement. The essential elements of reforms in this 
paradigm are threefold. First, a set of clearly defined student 
performance goals for schools to strive towards (usually content 
standards). Second, an accountability system comprising a set of 
incentives for schools or districts to achieve the standards and 
accompanying penalties for failing to move towards them (rewards 
and sanctions). And third, greater autonomy for districts and schools 
make decisions that will enable them to improve instruction and 
achieve the standards (Fuhrman, 1999; CPRE, 1996; Fuhrman & 
O'Day, 1996). 

While earlier reforms were characterized by either a "top- 
down" (i.e. mandates) or "bottom-up" (i.e. local control) approach, 
standards-based reform combines both approaches to enable states and 
districts to define the focus and expectations for educational outcomes 
and to hold educators accountable for meeting these aims (Fullan, 
1994). At the same time, policy makers recognise that instructional 
improvement needs to be motivated and developed at the school and 
classroom level and they are developing ways to give schools and 
teachers increased autonomy to make decisions that affect student 
learning. 

In this article we ask the question: What are the effects of 
increasing school-level autonomy and accountability in the context of 
standards- based reform? We investigate this question by exploring the 
experience of the school district of Cincinnati, Ohio's efforts to expand 
the autonomy of schools and teachers and construct a framework of 
accountability within the context of a broader standards- based reform 
initiative. The district's efforts to increase autonomy at the school and 
classroom-levels is focused around a reform called team-based 
schooling in which teachers are organized into teams of three to five 
teachers who take responsibility for a group of students over multiple 
years. The expectation is that teachers know best how to serve the 
needs of their students and should therefore be given greater flexibility 
and authority to make decisions that affect their students' learning. 
Teaming is part of a broader standards-based reform movement in the 
district featuring an explicit set of achievement targets for schools and 
rewards and sanctions tied to a school's success or failure in achieving 
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their goals. In this article we describe some of the consequences, both 
intended and unintended, which arise as the theory of increased 
authority and accountability plays out in the classrooms of Cincinnati's 
public schools. 

In section II, we set the context for our analysis by briefly 
describing the track record of site-based management in general and 
team-based schooling in particular. In section III we describe the 
design of Students First, the Cincinnati Public Schools' ambitious 
standards-based reform initiative designed to increase local autonomy 
within a framework of accountability. In section IV we analyze the 
consequences, both intended and unintended, of increasing local 
autonomy. Section V explores the effects of expanded accountability, 
both internal and external. We conclude by summarizing the findings 
of our research and discussing the implications in relation to the trade- 
off between increased accountability and increased autonomy on 
which standards-based reforms like teaming are based. 

Method 

The source of the data for this article come from an evaluation 
of team-based schooling in Cincinnati, Ohio being conducted for the 
Cincinnati Public Schools by the Consortium for Policy Research in 
Education (CPRE) at the University of Pennsylvania. Since 1997, 
CPRE has been documenting the evolution and effects of team-based 
schooling in Cincinnati. This article draws primarily from the first two 
years of our evaluation and is based on five data sources. First, the 
CPRE research team conducted extensive fieldwork in the district. 
During the first year of the evaluation we visited all of the eight team- 
based schools and interviewed administrators, members of the 
Instructional Leadership Team, and the full membership of a sample of 
16 teams. In 1998-99, the second year of teaming, the research team 
spent four to five days in each of the 20 team-based schools and 
interviewed the full membership of a sample of 41 teams. 

The second data source for this article is the survey we 
conducted each year with all faculty in both team-based and non-team- 
based schools. The response rate varied from 81 to 87 percent each 
year respectively out of a total of approximately 2,500 faculty. Survey 
results enabled us to compare team-based with non-team-based 
schools and has allowed us to examine the longitudinal effects of 
teaming. 

Third, we conducted interviews with the leaders of the 
Cincinnati education community including leaders of the Cincinnati 
Public Schools (CPS), the Cincinnati Federation of Teachers, CPS 
school board, and the Mayerson Academy which is the key provider of 
professional development to support the implementation of team-based 
schooling. Interviews focused on leader's perceptions of what team- 
based schooling would accomplish, the factors that influenced its 
implementation and impact, and the progress of the reform. 

The fourth data source was student achievement, attendance, and 
discipline information provided by the district's Office of Research. 
Data were analyzed using hierarchical linear modeling (HLM) to 
examine the relationship between teaming and student achievement at 
the school and team level. Finally, we observed Interschool Council 
Meetings, attended team-based schooling workshops run by the 
Mayerson Academy and related initiatives produced by the district and 
union. 
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Our purpose is not to report in detail the findings of CPRE's 
three year evaluation of the team-based schooling initiative in 
Cincinnati. Those interested in further details can read the evaluation 
reports (Supovitz & Watson, 2000; Supovitz & Watson, 1999; 
Supovitz, 1998). Rather, r we focus here on the implications of the 
Cincinnati experience with respect to the trade-offs and tensions 
inherent in expanding autonomy in a tighter accountability framework 
within the context of standards-based reform. 

II. The Trend Toward School-Based Management 

Many districts have attempted to increase local autonomy based 
on the theory that school personnel are most intimately knowledgeable 
about the best way to educate their students. Mohrman and Wohlstetter 
(1994) described various types of devolved management structures 
that have been adopted in the past under the general rubric of school- 
based management (SBM). SBM requires new forms of governance 
and management structures within districts and schools, structures that 
are designed to change the decision-making processes and relations of 
power. This reform movement operates under the belief in the 
ineffectiveness of locating power at the top of a school system, where 
it is furthest from school faculty who are closest to, and therefore most 
able to influence students. SBM is intended to create structures that 
support site-based decision-making so that school-based 
administrators and teachers have greater control over the decisions that 
affect student learning. SBM is not a new idea, having been 
implemented in various guises throughout the 1 980s and 1 990s 
(Weiss, 1992). But what is different in this latest incarnation is the 
melding of local autonomy with a system of accountability (Fullan, 
1994). 

Newmann and Wehlage (1995), in a large study of 1,500 
schools, noted that successful schools tended to create a professional 
community with shared purpose and high levels of collaboration. They 
noted that successful schools required the authority to act, and had 
high levels of autonomy over curriculum, school policies, hiring, and, 
in some cases, their budget as well. Surrounding this cocoon of 
autonomy, they also cited the importance of external support in the 
form of high standards for student learning. 

Chub and Moe (1990) argue that private school teachers 
experience higher levels of autonomy than their public school peers 
and that this is a key reason why private schools are able to more 
successfully promote student achievement. In a comparative study of 
teachers and principals in both private and public schools that set out 
to test Chubb and Moe’s thesis. Glass (1997) found that both 
participants said they experienced high levels of autonomy. However, 
they also found that teachers in private and public schools were also 
subject to a number of factors that mitigated and shaped their 
autonomy in important ways. Thus, the autonomy experienced by 
teachers was more complex than typically assumed and "challenges 
the myth that teachers and principals in private schools enjoy 
autonomy and freedom from democratic bureaucracy that their public 
school counterparts do not" (p.l). 

Other studies also question whether SBM actually increases 
school and teacher autonomy. Hannaway (1993) presents a critique of 
the existing assumptions on which decentralization is based, 
assumptions that are derived from private sector modes of production. 
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Hannaway argues that decentralization is based on the assumption that 
teachers' work is traditionally subject to a high level of control and that 
the limited autonomy of teachers serves to limit their ability to 
instigate the kinds of change and innovation necessary to promote 
student achievement. Decentralization and (so-called) greater teacher 
autonomy aims to give teachers greater control over how they teach. In 
contrast, Haimaway concludes that, "teachers in successful 
decentralized districts work under conditions where organizational 
controls over their behaviour are in fact high relative to what we would 
expect in traditionally organized schools. Indeed, the discretion of 
school-level actors in many decentralized systems may be far more 
restricted than the discretion of school-level actors in traditionally 
organized systems" (1993, p. 139). In other words, she contends that 
decentralization decreases the amount of autonomy that teachers have 
to decide what and how to teach and that this occurs through the 
increased surveillance of their work at two levels: by their peers and 
by the district through tighter accountability measures. Merely 
providing increased opportunities for interaction between teacher 
professionals will not necessarily result in productive work and 
change. 

The literature that examines the development and 
implementation of various forms of SBM also points out that the key 
aim of such designs should be to improve the quality of teaching and 
learning within schools. In other words, for SBM to be successfol, it 
must not be merely a structural reform, but one that results in 
instructional change at the classroom level. Darling-Hammond (1996) 
described in some detail four New York schools that have successfully 
introduced decentralized, site-based management reforms as a means 
of creating more consensual decision-making and greater autonomy 
for schools and teachers with the purpose of approaching teaching and 
learning in distinctive ways. The success of these schools, argues 
Darling-Hammond, is that the primary focus of their changes is to 
create the opportunities for improved teaching and learning. 

Team-Based Schooling 

One of the particular school-based management strategies that 
have been developed in an attempt to provide an instructional focus is 
the use of teacher teams. Teacher teams are designed to enable 
teachers to have greater involvement in the management and 
governance of their school and are also intended to facilitate 
instructional change and innovation as teachers work more elosely 
together to learn from each other. Educators have been experimenting 
with the "bold new venture" of team teaching since the 1960"s 
(Thomas, 1992; Beggs, 1964), but with little success (Thomas, 1992). 
The early failures of teaming were attributed to a lack of 
organizational support, planning time, and role conflict (Hargreaves, 
1980; Cohen, 1.976). 

The last decade has seen a resurgence of interest in teaming 
largely as a result of the apparent success of production teams in 
private enterprise and the attempts by some administrators to apply 
teaming to schools (Mohrman & Wohlstetter, 1994). Darling- 
Hammond (1994) gives an account of a number of schools who have 
used teaming successfully in order to create more consensual decision- 
making and greater autonomy for schools and teachers. More 
importantly, however, they have used teaming to enable more inter- 
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disciplinary learning for students and to also promote teacher learning 
which is essential for instructional change. 

Friedman (1997) offers a detailed account of one teacher team 
that developed an innovative vocational design within an urban high 
school. Friedman argues that teaming requires changes to traditional 
teaching roles and school structures and that these changes are more 
complex than those typically envisioned by the proponents of teaming. 
The success of teaming therefore appears to depend on it not being 
merely an organizational or structural reform, but one that promotes 
and supports changes in how teachers teach. Friedman's research alerts 
us to the need to consider not only the assumptions and intent behind 
team-based schooling initiatives, but also the existing structures and 
cultures of schooling that they are enacted upon. Teachers are not 
passive recipients of reform, but they actively negotiate and mediate 
policy in a range of ways. It is not so much that policy is enforced 
from above, but rather that teachers enact policy in a range of ways 
that result in unintended and intended consequences. As Friedman 
observed: 

... few advocates have inquired seriously into the team 
concept and exactly how it fits with school practice. This 
lack of conceptualization is particularly serious in light of 
the failure of the initial team-teaching movement of the 
1960s, which has been attributed to a lack of fit between 
the team concept and the role of the teacher, the 
organizational structure, and the cultural norms of 
contemporary schooling (1997, p. 335). 

This report enables us to undertake the kind of enquiry that Friedman 
argues is necessary if we are to further our understanding of the effects 
of the implementation of team-based schooling. If teaming is to be 
effective, it must result in instructional innovation and improvement. 
For this to occur, such a reform needs to attend to the relations of 
power that operate within schools. 

III. Autonomy and Accountability in The Cincinnati 
Public Schools 

Cincinnati, Ohio is an urban school district with about 50,000 
students in 79 schools. About two-thirds of the students in the district, 
of which 70 percent are African-American and 28 percent are White, 
are on lunch assistance. Cincinnati is the forty fifth largest city in the 
United States. In the 1996-97 school year the Cincinnati Public 
Schools (CPS) adopted an ambitious and broad-based reform plan 
called Students First. The stated goals of Students First were for all 
students to meet or exceed high academic standards, to have safe and 
orderly school learning environments, and to satisfy the needs of their 
"customers" - students, parents, and taxpayers. As a central part of its 
strategic plan, the CPS designed a form of school organization known 
as team-based schooling. In essence, the idea behind team-based 
schooling is that higher student achievement will result from 
decentralizing decision-making about instruction and resource 
authority to teams of academic teachers. The teams are to focus on the 
district's academic and behavioural standards, to collaborate amongst 
themselves as well as with parents and community members, and to be 
held collectively accountable for their students' achievement over time. 



httn ://enaa.asu. edu/enaa/v9n3 2 . html 



155 



10/29/01 




EPAA Vol. 9 Autonomy and Accountability in the Context of Standards-Based Reform Page 7 of 23 



As stated in the district's strategic plan, the organization's reform goal 
was to become a, "high quality education system that is decentralized 
and held accountable for results." 

The "heart of the system" according to the former 
superintendent, is team-based schooling. Team-based schooling is 
written into the contract between the CPS and the Cincinnati 
Federation of Teachers, ratified in March 1997. The contact sets out 
the requirements for the composition and function of teams, as 
summarized below: 

• Teams will be comprised of 3-5 core subject academic teachers 
who will stay with a group of students for at least two years. The 
teams will be organized by the gateway grades K-3, 4-6, 7-8 and 
9-10. 

• Teams will develop a curriculum and instructional methods and 
materials consistent with a school's program focus. They will 
also decide how to schedule and group their students. 

• Teams will take responsibility for all students they serve and 
will work to ensure that they meet the district and school 
learning objectives. 

• Teams will control funding for instructional supplies, materials, 
and personnel. 

• Teams will stay together for several years in order to ensure 
maximum benefits from collaboration and longer term relations 
with students. 

• Team-based schools will be governed by Instructional 
Leadership Teams (ILT), comprised of team leaders, the 
principal, two parents, and two non-teaching school staff. The 
ILT will attend to academic decisions and control most non- 
personnel budget areas. 

The district planned to "roll out" the team-based school concept 
across the district, beginning with the eight schools that were selected 
from a pool of those that applied in 1997-98. Twelve additional 
schools became team-based in the second year (1998-99), and twenty 
schools in the third year (1999-2000). At the time of writing, in the 
third year of the reform, the implementation has proceeded as planned 
and approximately half of the district's schools are team-based. 

It must also be noted that teaming is not a new phenomena in the 
Cincinnati Public Schools. Our 1999 teacher survey data show that in 
Cincinnati 79 percent of elementary teachers, 73 percent of middle 
school teachers, and 45 percent of high school teachers who were not 
in formal team-based schools reported that they informally teamed 
with at least one other teacher. Wiat is different is that team-based 
schooling creates a formal structure in schools that explicitly vests 
power in teams of teachers to make instructional decisions for 
students. 

At the same time, the state and district had accountability 
systems in place that r^iked the performance of schools and attached 
consequences to their performance. In Ohio, districts are rated 
annually on a set of 27 indicators, of which 25 are student achievement 
tests as well as student attendance and graduation rates. Based upon 
these ratings, districts are put into four categories: effective, 
continuous improvement, academic watch, and academic emergency. 
These categories are highly correlated with the socio- economic status 
of districts. Districts in academic emergency have five years to move 
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out of the category or are threatened with state takeover. CPS is in the 
academic emergency category, but despite one of the highest levels of 
poverty of any district in the state, it is not the lowest performing 
district. 

The district's accountability system is called the School 
Accountability Plan. In it, targets are set for each school in six areas, 
which are very similar to the state indicator areas. Unlike the state 
system, targets differ by school, which are set based upon the results of 
the previous year. Targets are set so that if all schools meet their 
targets, than the district will meet its target. Based on their 
performance, schools are rated as either a school incentive award 
winner, an achieving school, an improving school, a school in 
intervention, or a school under redesign. Redesigned schools can, and 
have been, reconstituted. Principals' pay raises depend partially on the 
performance of their school. 

The theory of how teaming works 

In interviews conducted in the fall and early winter of 1997 with 
14 leaders of the Cincinnati education community, the leaders 
described how they expected teaming to work. The district leaders 
emphasized improving student achievement and the quality of the 
educational experience as the overall expectation of team-based 
schooling. Leaders hypothesized that team-based schooling would 
impact the district in a variety of ways. The influences that were 
mentioned can be loosely organized around four inter-related themes: 
decentralized control, more focused curriculum and instruction, a more 
student-focused school culture, and increased accountability. 

District leaders described how they expected teaming to shift 
decision-making about curriculum and instruction from the district to 
schools to teams of teachers, giving school staff a greater role in 
critical decision-making about their work and greater control of their 
budgets. They saw this devolution of authority driving related changes 
in the central office while giving schools greater autonomy from the 
central office. 

Many leaders therefore stressed that team-based schooling 
would increase teachers' focus on curricular and instructional issues. 
They felt that attention to the curriculum standards would increase, 
improving curricular planning and alignment. They expected that 
teachers would make more fine-grained decisions about grouping of 
students, resulting in more individualized instruction. They also 
expected that looping (teachers staying with the same group of 
students over multiple years) would push teachers to expand their 
curriculum knowledge. In these ways decentralization, along with 
increased accountability, would lead schools and teams to allocate 
their resources more productively. 

Leaders also envisioned a series of influences that can loosely be 
called a more student-focused school culture. Under this element of 
their vision, a series of new relationships and norms would develop in 
team-based schools. Teachers would get to know students better, 
would analyze student achievement data and would be better informed 
as they designed instruction to more effectively meet students' needs. 
One leader stressed that the teams would form communities, providing 
a greater reflection of the democracy we live within. 

The local leaders also felt that team-based schooling would 
increase the accountability of teachers. They described how the new 
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teams would give teachers a greater sense of students' 
accomplishments and encourage them to take more responsibility for 
the progress and success of individual students. Since each of the 
teams would be responsible for preparing students for one of the 
gateway grades at which promotion benchmarks must be met, all 
teachers would share the responsibility that had previously rested more 
heavily on those teachers assigned to the gateway grades. Further, it 
was envisioned that a culture of competition would emerge in the 
effective team-based schools, propelling teachers to higher quality 
instructional levels. Teaming therefore has the potential to meet the 
demands of increased autonomy and accountability at the school level 
through decentralization, rather than through increased centralization 
as has been a feature of earlier reforms. 

Reforms are seldom implemented in isolation and other 
initiatives and events often have unintended effects. The 
implementation of team-based schooling in Cincinnati was no 
exception. Three events occurred in Cincinnati which both directly 
altered the implementation of Students First, and more subtly 
influenced the environment within which the reform was unfolding. 
First, one of the major architects of Students First, J. Michael Brandt, 
retired as Superintendent after the first year of team-based schooling 
and was replaced by Dr. Steven Adamowski, although the Board 
restated its commitment to Students First. Second, the district 
experienced severe budget cuts and schools were forced to reduce staff 
and resources. Third, in the 1 999-2000 school year, the district and 
union went through a prolonged contract renegotiation. Additionally, 
during the time of the initiative, the district announced major 
organizational changes, including a reorganization to K-8 schools, a 
plan to shift to open enrollment for high schools, the adoption of a new 
facilities plan which targeted which targeted some schools for eventual 
closure, the expansion of charter schools. This confluence of events 
and developments generated considerable anxiety and undermined 
morale among teachers. The overall climate of rapid change and 
uncertainty disturbed some teachers' confidence in the stability of their 
teams and in the future of the team-based reform. 

IV. Expanding the autonomy of teachers in schools 

As we have discussed, the team-based schooling reform 
envisages that schools, teams and individual teachers will have greater 
control over how they teach while also being held increasingly 
accountable for the achievement of their students. This notion of 
increased autonomy raises a series of important issues that concern the 
definition and limits of autonomy and the authority that resides at 
different levels within schools - including teachers, team leaders, and 
the principal. These issues also played out between schools and the 
district. That is, the increased autonomy of teaming created a whole 
new set of relationships within schools that implicitly modified the 
existing traditional hierarchy. 

The traditional hierarchy of schools has a principal (and vice- 
principal in larger schools) sitting on top of a traditionally flat 
organizational structure. Schools often have a large number of 
committees to make instructional decisions, but formal authority is 
vested in the principal. Team-based schooling changed the 
arrangement such that teams of teachers are formally made responsible 
for instructional decisions about their students. Teams are led by team 
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leaders while the school is led by the Instructional Leadership Team 
(ILT), comprised of team leaders and led by the principal. The ILT is 
officially responsible for making school-wide instructionally related 
decisions. This new, more multi-faceted organizational structure 
created uncertainty as the members sought to clarify the extent and 
limits of their autonomy and authority. In the course of our fieldwork 
these concerns were at the forefront for school faculty as they 
struggled to implement team-based schooling. In this next section, we 
explore in some detail the issues around autonomy and authority that 
arose in schools as their faculty implemented the reform. 

The role of the team leader 

The key purpose of teams is to establish a structure that provides 
teachers with increased opportunities to work together to improve 
instruction. The collective bargaining agreement between the teacher 
union and the district states that each team shall have a paid team 
leader and describes the procedures for their selection. However, there 
is little specification about the team leader's role or the authority they 
have to make decisions that affect the members of their team. In some 
teams, members naturally deferred to their team leader to organize the 
time and resources of the group while in others this process was 
collaborative. There were a small number of teams in which the team 
leader's efforts to exercise authority created friction amongst the team 
members. 

In our interviews with a sample of members of 41 teams in the 
second year of teaming (1998-99), a number of team members and 
leaders discussed the problems caused by the unclear responsibilities 
of team leaders. For example, in a team in a K-8 school, some team 
members did not accept the team leader's authority, especially the 
teachers who knew they would not be returning to the school the 
following year. Two of the four teachers had requested and received a 
transfer to another school and a third did not have his contract 
renewed. As a result, the team leader said the team was dysfunctional. 
"The only part where there is really teaming academically is with [the 
special education teacher]. We share responsibility for his 
developmentally handicapped kids and my grade 7 to 8 kids. That 
works great, but it isn't any different from what I was doing before 
teaming." 

In another elementary school, there was a serious conflict 
between the team leader and one of the team members. The team 
leader said the conflict was caused by a personality issue, but the 
teacher said it was due to their different teaching styles. The teacher 
did not want to follow the team leader's advice on instruction and she 
said, "The team leader always wants things her way." The team leader 
commented, "Lots of teachers have different visions but teams need to 
have a united vision. We don't talk about a philosophy or teaching 
styles as a team." These examples raise important questions about 
team leaders' capacity and authority to propel their teams to higher 
functioning levels. 

This lack of definition of the team leader's authority was of 
concern because in some cases it prevented teams from engaging in 
instructional change, a central aim of team-based schooling. An 
elementary school team leader described how the team members found 
it hard to share student work samples and resisted doing this even 
though they agreed, in theory, that it was an important practice. One 
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team leader had asked to observe another teacher's class, but it did not 
happen. "I could have insisted on certain things, like sharing examples 
of students' work, but I don't see that as my role. I don't feel I have the 
right to challenge teachers about their practice. My role is to set a tone, 
an expectation of being professional." 

Thus, in theory, while the teams were given autonomy to make 
decisions about instruction, in practice, the decision-making process 
required someone with institutional authority to facilitate the process. 
This lack of definition of autonomy and authority was also evident 
within the Instructional Leadership Teams (ILT). 

The role of the ILT and the principal 

The role of the ILT in each team-based school is to develop, 
review and evaluate the instructional program and to monitor and 
improve school operations and procedures that impact on instruction. 
The ILT is also to develop and monitor the school budget and to 
oversee the formation of teams and can decide on the process by which 
it makes decisions, for example by vote or consensus, and faculty will 
also be required to approve the ILT recommendations by majority 
vote. The ILT therefore, as its nomenclature implies, is to provide a 
forum for decisions to be made about instruction and those decisions 
are then to be disseminated and implemented at the team-level. Our 
observations of ILT meetings and interviews with ILT members and 
principals in the second year of teaming highlighted the difficulties in 
implementing a devolved leadership structure. 

The design structure of the ILT represents a significant shift in 
the way decision-making power is distributed within a school. 
Authority for the overall direction of the school traditionally resided 
with the principal but the team-based model is intended to shift 
authority for making some decisions down to the team level. The 
introduction of the more democratic decision-making through the ILT 
represents a major change from more hierarchical school power 
structures. 

Although the ILT is intended to be run in a consensual way so 
all members are able to make an equal contribution to the decision- 
making process, the reality is that the differential power relations 
among the various members can undermine the decision making 
process. We found that the principal (or someone else on the ILT) had 
to actively push for shared decision-making because the traditional 
hierarchical culture of schooling predisposes teachers to defer to the 
principal. 

For some principals, moving toward more consensual decision- 
making processes was a difficult transition to make. The principal, in 
theory, is an equal member of the ILT with no more authority than 
anyone else, but the principal's role in monitoring and evaluating staff 
performance may m^e teachers reluctant to challenge the principal on 
an issue. Principals can exercise their power advantage or may 
suppress differing views from being expressed during ILT discussions. 
Some principals were described as effective at encouraging consensual 
processed while others were described as being adept at ensuring their 
ideas were prioritized and implemented. 

Principals promoted their own agendas in direct ways, including 
actually setting the agenda and facilitating the ILT meetings, and in 
indirect ways, which some faculty members described as manipulation 
behind the scenes. In our interviews in the second year of teaming, one 
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ILT member stated, "The principal stonewalls and dominates the 
conversation." Furthermore, while most principals paid lip service to 
the notion of consensual decision-making, there were some whose 
leadership styles were in direct and stated opposition to this. One 
principal rarely attended the ILT meetings and made some decisions 
without consulting the ILT at all. As a result, several ILT members 
said they would be stepping down as team leaders because they no 
longer wanted to be part of the ILT. In several schools, principals 
made decisions that undercut ILT consensus. 

Faculty and administrators in the team-based schools were 
adopting a range of relationships as they sought to navigate the 
transition from a traditional hierarchical culture to the more 
democratic culture reflected in teaming. School leaders at one end of 
the spectrum were remaining traditional. As one team leader 
described, "A TBS [team-based school] principal needs to understand, 
explain, share. They need to be willing to let go. [Our principal] 
doesn't share with us what we need to know and it leads to frustration." 
At the other end of the spectrum, a principal at another school was 
adopting democratic decision-making to the extent that he was not 
willing to make any decisions himself. A team leader in the school 
commented, "The principal is delegating everything and not being a 
leader." Most principals were seeking a middle ground. As one 
principal described, "I try to empower people but they have to 
understand that this is a hierarchy and I am the principal." 

Ideally, the ILT is designed to ensure that decision-meiking 
authority does not reside solely with the principal. However, given the 
complex power relations that operate within schools and the control 
that the principal has over the retention and promotion of staff, the 
reality is that most principals still maintain a high degree of control. 
Some teachers seemed pleased to have a principal they perceived as a 
"strong" leader who could maintain control of the school. Other 
teachers questioned whose responsibility it was to ensure that 
principals in team- based schools operated in ways consistent with the 
team-based philosophy. 

Role of the team vis-a-vis the ILT 

While there were issues of authority within the ILT and between 
the ILT and the principal, there were also issues of authority raised in 
relation to the decisions being made by the ILTs concerning the 
division of authority between the ILT and the teams. When 
interviewed during the second year of teaming, some teachers said 
they were confused about the role of the team compared to the role of 
the ILT. In several cases, team leaders felt that the ILT was overriding 
decisions that had been made by teams, decisions that they felt the 
teams should have been able to make. This was exemplified by an 
elementary school where there were a range of opinions about the 
benefits of multi-age classes and looping (the practice of students 
remaining with a teacher for more than one year). A team leader in the 
school commented, "We are still trying to work out the mechanics as 
far as what teams can or cannot do. This year, the ILT decided either 
looping or multi-age — one or the other. The majority of the school 
doesn't want to do either one." This example raises the question of 
whether there should be some limitations on what teams can decide 
and what the ILT has the authority to decide. 

Differences of opinion about the levels of autonomy and 
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authority within a school need to be discussed and resolved if team- 
based schools are to function effectively. A team leader in another 
school commented, "There are some rumbles that the ILT is trying to 
run everything but teams should be represented by team leaders so this 
shouldn't be the case." An ILT member in the same school 
commented, "The ILT makes decisions about programs, budget items, 
etc. Teams can make decisions within their teams. Yet for discipline, 
the encouraged school wide policy is what is best for children." 

The role of the district in relation to the school 

The lack of clarity within team-based schools about the limits of 
their authority and autonomy was also reflected at the district level. It 
was apparent from our interviews with leaders in the Cincinnati 
education community in 1998-99 that the district had not yet worked 
out what decisions the school should have the authority to make, and 
which decisions were the purview of the district. For example, there 
were a number of examples of the district encouraging schools to 
make their own decisions about school reform models or curriculum 
resources and then telling the school that their decision was not 
acceptable. 

We also noted some differences emerging in district leaders' 
views about how much autonomy should be given to schools. While 
they generally agreed at a conceptual level that schools and teachers 
should be given increased autonomy to make decisions aimed at 
improving student achievement and should be held more accountable 
for the results obtained, there were important differences of opinion 
about what this meant in practice. Our interviews suggest that the 
views of district policymakers on this issue fall along a continuum. At 
one end of the continuum are those who believe that schools should be 
given as much autonomy as possible to decide what and how students 
should be taught, and which reform models to use. In this view, 
autonomy should be constrained only by the standards (promotion 
standards and proficiency tests) that are mandated by the state and the 
district. How school staffs reach these standards is up to them. One 
leader in the district summed up this position, "Let the standards shape 
what schools do, but let them find their own way." Furthermore, some 
of these proponents of increased school autonomy would like to see 
schools freed from some of the current contract requirements, such as 
those that limit meeting times and require vacancies to be filled from 
the surplus pool instead of allowing schools to recruit their own staff. 

At the other end of the continuum are those who favor granting 
less autonomy to schools and teachers. For example, they believe that 
schools should choose only those whole school reform models 
previously approved by the district and that teachers should not be 
allowed to adopt curricular materials that have been tried and failed in 
several other places in the district. Those in favour of constraining 
autonomy at the school level believe that setting academic standards 
for schools is not sufficient, and contend that schools should teach a 
district-approved curriculum and that students in all schools should 
follow a core academic program. 

These differing views of autonomy are associated with different 
views of team-based schooling. Those favoring maximum autonomy 
are content to permit more variation in how ILTs function, how teams 
are structured, and how resources are allocated by the schools. Those 
who favor some constraints believe that the district should ensure that 
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team-based schooling is correctly implemented and that the structures 
are faithful to the original design. In practice, this has raised a number 
of questions of how the district should respond to issues. For example, 
a middle school team of a dozen teachers who want to stay together as 
a team, regardless of the suggested team size of 3-5 teachers; or a 
principal who ignores the unanimous recommendations of the school's 
ILT. 

As teachers and school administrators sought to define the limits 
of their authority, they were doing so in the context of a high-stakes 
accountability framework at both the state and district level. In the 
next section we discuss the issues that arose in response to increased 
accountability in the context of team-based schooling. 

V. Increasing accountability 

As previously described, the district's reform initiative. Students 
First, contains a school accountability plan in which targets are set for 
each school based upon indicators of progress, primarily achievement 
scores in the five core subjects, as well as student attendance, dropout 
rates, and graduation rates. Progress is defined differently for each 
school, depending on its previous year performeince on these 
indicators. Rewards and sanctions are meted out based upon school 
performance each year. Thus the "teeth" of the accountability system 
are at the school, rather than the team-level. But this does not mean 
that teams do not feel the bite of accountability. 

The pressures of internal accountability 

Abelmann and Elmore (1998) make the distinction between 
systems of accountability that are external and internal to schools. In 
their conceptualization, state and district accountability systems are 
external. But internal accountability systems operate in equally 
powerful ways and are made up of individual responsibility and 
collective expectations that together shape they ways that people 
account for their actions. They found that, "strong expectations can 
influence and shape what a teacher. . .feels responsible for in his or her 
work " (p. 17). Similarly, our survey research found that teachers in 
team-based schools had significantly higher levels of collective 
responsibility and reported higher levels of involvement in school- 
related decision-making (Supovitz& Watson, 1999). 

Survey data from the second year with all faculty in team-based 
schools revealed that teachers were overwhelmingly positive about 
working together on teams. Over 90 percent of team members reported 
that they worked well together as a team to do what was "best for 
kids." A similar percent felt comfortable voicing concerns with team 
members. In interviews, several teachers described how they valued 
the relationships they were able to develop with other team members. 

A team member in an elementary school explained, "I like the concept 
of team-based. You are working with the same group of kids. I like the 
communication and feedback you get." In a middle school, a team 
member commented that teams enabled improved communication that 
helped them to better address the needs of students. "We have more 
adult communication via teaming. The team spends a lot of time 
brainstorming on how to mix and match kids." 

Teaming also tended to increase the pressure that teachers put on 
each other. In some cases this was a constructive force, but in others it 
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served to undermine working relationships between teachers. This 
resulted in higher levels of stress and increased tensions between 
teachers in team-based schools. Survey results from the first two years 
of CPRE's evaluation indicated that trust levels between teachers were 
significantly lower in team-based schools in comparison to non-team- 
based schools, and that trust levels declined from the year before 
schools implemented teaming to the end of their first full year in the 
initiative (Supovitz & Watson, 1999). These results suggest that the 
introduction of the formal mechanisms of teaming had surfaced issues 
which were avoidable when teachers did not have to work so closely 
together. As one team leader commented in the second year of 
teaming, "There is a cultural shift going on in the school. Teams are 
more vocal than before and people that used to be passive are more 
assertive." 

Another consequence of increased interactions between teachers 
around instructional issues was higher levels of conflict. For teachers 
who may be used to working with other teachers on discipline or 
curriculum committees, but who are none-the-less used to closing their 
classroom doors for instruction, teaming can increase opportunities for 
disagreement. On our survey in the second year of teaming, over half 
(53%) of the team members reported conflict between members of 
their team. Personality clashes were the greatest sources of conflict, 
accounting for 67 percent of the conflicts. Disagreements over the 
equitable distribution of work, student discipline procedures, 
educational philosophy, and disagreement over curriculum and 
assessment issues were also common. 

Of course, conflict in itself is not necessarily negative because it 
may be the product of increased communication among teachers and 
more engagement in critical discussion of practice and philosophy. 

Our interviews with team members suggested that conflict was a 
problem only when it remained unresolved and, therefore, undermined 
the relationships within teams. Our survey also included a series of 
questions for the team members who reported having conflict in their 
teams about how they attempted to resolve it. Over half (57%) of the 
teams reported that they resolved the conflict amongst themselves but, 
of the 53 percent of team members who said they experienced conflict, 
30 percent said that the conflict remained unresolved. Overall, this 
means that approximately 15 percent of the teachers were experiencing 
unresolved conflict on their teams, a percentage that corresponded 
closely to our interview data. 

While team relationships are something that, with the 
appropriate support and skills, can be improved and strengthened, 
there are other factors that affect the operation of teams and student 
achievement which teams have little or no control over. In the course 
of our research, teacher and student turnover emerged as significant 
factors affecting teams. 

External accountability 

In general, there was a certain amount of resistance from 
teachers to being held accountable for the performance of their 
students. In our 1998-99 survey of school faculties, over 50 percent of 
the teachers in the team-based schools did not believe it was fair to 
hold teams accountable for the achievement of their students. Our 
fieldwork helped us to understand some of the reasons why teachers 
felt uncomfortable with this responsibility. 
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One reason that teachers resisted accountability was high rates 
of both student and teacher mobility. The design of team-based 
schooling states that teams will be composed of groups of teachers 
who will be held jointly accountable for the achievement of a group of 
students for at least two years. This model assumes that the 
composition of the teams, in terms of students and teachers, will 
remain stable to ensure maximum benefits from collaboration and 
longer-term relationships with students. But teachers were quick to 
point out the high mobility of their student populations and that high 
student mobility rates made it unreasonable for them to be held 
accountable for student achievement. How could they be held 
responsible when many of their students were with them for relatively 
little time? 

Second, teachers felt that teacher mobility impeded their ability 
to work together to meet the needs of students. Our survey with faculty 
in team- based schools in the second year asked teachers if the 
membership of their team had remained the same over the school year. 
Less than one-third of teams reported stability (in terms of their 
teacher composition) from the first to the second year as team-based 
schools. Of the teams that did change, about one-third changed 
significantly and two-thirds experienced minor changes. The reasons 
for the turnover among team members varied. In some cases they were 
due to decisions made by team members to leave the teaching field, 
take long-term leaves, or transfer to other schools or positions. But in 
many cases, team instability was due to decisions made at the district 
and school level that affected the school's staffing and resulted in 
changes to team composition. 

We also found that while most teachers felt their autonomy has 
increased, at the same time, teachers also reported that they 
experienced significant restraints on their autonomy caused by the 
introduction of standards-based accountability. Our work suggests that 
the requirements of standards-based accountability, which brought 
with it performance standards, pacing guides, curriculum materials and 
"standards into practice" protocols for those schools that failed to meet 
their targets may counteract efforts to increase local autonomy. 

How were teachers using their increased autonomy and 
accountability? 

The litmus test for the effectiveness of the team-based schooling 
reform is whether increased autonomy and greater accountability have 
resulted in the kinds of instructional improvement that leads to 
increased student achievement. The results of our evaluation show that 
when teams are structured in ways envisaged by the reform, and when 
they are engaging in practices that teaming was designed to enable 
such as plarming lessons together, reviewing student work together, 
co-teaching classes and observing each other teach, then there is a 
positive impact on student achievement (see Supovitz & Watson 2000 
and Supovitz & Watson 1999 for more detailed discussion of the 
methodology used). However, we also found that implementation 
varied and there were very few teams that were using the kinds of 
practices that were positively associated with student achievement. 
Even when teams were functioning well in terms of team meetings and 
team relationships, very few teams had begun to engage in 
instructional improvement by changing the way they were teaching. 
While our research has shown that there were a number of factors such 
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as resource constraints, lack of definition of authority, and student and 
teacher mobility that undermined the effectiveness of team-based 
schooling, the small number of teams that were responding to 
increased autonomy and greater accountability by attending to 
instructional improvement is a significant concern. 

VI. Discussion 

The purpose of this article has been to explore the "lived reality" 
of team-based schooling— a standards-based reform that aims to 
increase autonomy and accountability at the school-level. Increased 
autonomy is to be provided by giving schools and teams of teachers 
increased control over how they teach and over other factors that 
influence student achievement such as scheduling, curriculum and 
grouping of students. Increased external accountability is enacted in 
the form of state and district level performance targets. Internal 
accountability is enacted in the form of increased peer accountability 
within teams. This design assumes that holding schools and teams 
more accountable for student achievement, while at the same time 
giving them more autonomy, will enable them to promote student 
achievement. Our research enables us to make the following 
observations. 

First, implementing any reform requires the recognition that it is 
enacted into an existing culture. In this case, team-based schooling did 
not replace the existing culture and relations of power within schools. 
Rather, faculty both accommodated and resisted various aspects of the 
reform so that the "new reality" in schools became a hybrid of both of 
these at times contradictory schooling structures. Although 
unarticulated, the success of the teaming design is predicated on a 
significant shift in relations at every level of schooling and this kind of 
shift, by necessity, will involve a change in the traditional relations of 
power by which the daily life of the school is structured. If not made 
explicit and attended to, these forces have the potential to subvert the 
kinds of changes on which teaming depends. 

Second, the lack of definition and clarity in the reform design 
about the limits of authority and autonomy that operate at various 
levels — from the district toough to individual teachers — hinders the 
ability of teachers, teams and schools to exercise the increased 
autonomy that team-based schooling was intended to provide them 
with. As our research has shown, this lack of clarity has sometimes 
resulted in conflict and/or in an inability to focus on the challenging 
work of instructional improvement. We recognize, however, that the 
reform process is a "work in progress" and through our research and 
through its own mechanisms, the district has been closely monitoring 
the reform and attempting to provide the kinds of guidance and 
resources that teachers are saying they need. 

Third, internal accountability is a powerful force. The pressures 
of collective responsibility unleashed by placing the responsibility for 
student achievement on teachers works in both constructive and 
destructive ways. On the one hand, teachers are forced out of isolation 
and are held increasingly accountable by their peers for their role in 
contributing to instructional improvement. On the other hand, as 
Cohen (1976) and Hargreaves (1980) have pointed out, expanding 
local responsibility is often accompanied by increased conflict. Our 
research indicates that conflict can have both productive outcomes, for 
example when it forces teachers to confront issues that impede 
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instructional coherence, and unproductive outcomes, when conflict is 
left unresolved and corrodes interaction and trust. 

Fourth, while external accountability mechanisms are premised 
on the assumption that teachers, teams and schools can be held directly 
accountable for student achievement, in reality there are a large 
number of factors over which they have very little control but which 
impact negatively on student achievement. These include such factors 
as student and teacher turnover caused by decisions made at the 
district-level in response to resource constraints and school 
restructuring. As Elmore (2000) argues, if policy makers and 
administrators are going to hold teachers accountable for certain 
outcomes, then they need to ensure that teachers have the capacity and 
resources to achieve those outcomes. Furthermore, while the design of 
team-based schooling promotes greater school and team-level 
autonomy, external accountability mechanisms constrain autonomy in 
very real ways. 

Thus far we have focused on highlighting the practical and 
relational issues that have arisen as teaming was implemented. But in 
the final part of this article we want to broaden the discussion to 
consider the trade-off between increased accountability and increased 
autonomy on which standards-based reforms such as team-based 
schooling are based. The assumption behind this trade-off are that 
teachers, teams and schools will respond to greater accountability by 
using their increased autonomy to engage in instructional 
improvement. Our research has highlighted some of the very real 
constraints that serve to limit autonomy and that also raise concerns 
about the extent to which teachers can be held directly accountable for 
student achievement in the way that standards-based reforms are 
designed to do. We have shown how these constraints hinder the 
ability of teachers and teams to engage in the challenging work of 
instructional reform. However, the question we want to raise here is 
whether — resource constraints and design problems aside — increasing 
the autonomy of teachers within the context of an external 
accountability system is the way to improve instruction. Or, to put it 
another way, do we have evidence that teachers were using their 
increased autonomy to engage in instructional improvement in order to 
meet the demands of the external accountability system? Our research 
shows that most teachers and teams were not. For the majority of 
teams, teaming is a structural, rather an instructional reform. That is, 
teaming can facilitate improved communication between teachers and 
allow them greater decision-making power over certain areas, but most 
teams do not use their increased autonomy and greater accountability 
to engage in instructional reform. 

Our research suggests that modifications need to be made to the 
ways in which autonomy and accountability are used within the 
context of standards- based reforms. Thus, the question is not so much 
whether teachers should be given more or less autonomy, or should be 
held more or less accountable. A more important question is how 
might autonomy and accountability be used to create incentives for 
teachers to engage in instructional improvement? One of the ways that 
this might occur is by modifying the accountability system. At the 
present time, schools and teams are primarily held accountable for the 
achievement of their students on standardized tests. However, as we 
have shown, there are some very real concerns about the fairness of 
holding teams and schools accountable on such a limited measure of 
school effectiveness. An alternative would be to hold schools, teams 
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and teachers accountable for their instructional practice so that they are 
rewarded for those aspects of their work over which they have direct 
control. In this way, the accountability system would encourage 
teachers and teams to focus on instructional improvement. 

A second way that teachers might be encouraged to engage in 
instructional improvement is by constraining teachers' autonomy in 
strategic ways. As we cited in the introduction, Hannaway (1993) 
pointed out that teachers in successful decentralized districts actually 
have less autonomy than teachers in traditionally organized schools. 
That is, while decentralization is typically seen as a means of 
increasing teacher autonomy, in fact, it constrains teacher autonomy 
by exposing teachers' practice to increased surveillance by their peers 
and by the district through the increased accountability measures. Our 
research shows that this is the case in team-based schools, but that 
increased surveillance leads to instructional improvement in very few 
teams. The challenge is to use increased autonomy and surveillance 
productively — in ways that result in instructional improvement. This is 
an important observation because while standards-based reforms ^lre 
designed to trade-off increased autonomy with greater accountability, 
our research suggests that increasing autonomy and accountability per 
se may not result in instructional improvement. Instead, it may be 
more constructive to design reforms that constrain autonomy and 
accountability in ways that require and enable teachers to engage in 
instructional improvement. 
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Abstract 

In 1991-97, the International Association for the 
Evaluation of Educational Achievement (lEA) undertook 
a Third International Mathematics and Science Study 
(TIMSS) in which data about the mathematics and science 
achievement of the thirteen year-old students in more than 
40 countries were collected. These data provided the 
opportunity to search for patterns of students' answers to 
the test items: which group of items was relatively more 
difficult (or more easy) for the students from a particular 
country (or group of countries). Using this massive data 
set an attempt was made to measure the similarities 
among country profiles of how students responded to the 
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test items. 

Introduction 

In the educational community, folklore has it that "The German 
education system is quite similar to that of Austria," or "All post- 
communist countries teach mathematics in the same way," and the like. 
Sometimes these statements are based on an analysis and comparison of 
national school structures, the curricula or textbooks. Is it really possible 
to measure the similarity between the countries? Usually, the 
phenomena of the similarity of the national educational systems is 
descriptive and subjective; their features are seldom measured and 
placed on a scale. Data from the Third International Mathematics and 
Science Study (TIMSS) provided the opportunity to search for patterns 
among nations in students' answers to the test items. (Notes 3 and 4) 

An attempt to group the TIMSS participating countries had 
already been undertaken by analyzing national curricula in mathematics 
and science (Notes 1 and 2). The countries were grouped by a priori 
geographic and economic conditions, or by investigating statistically 
some patterns in the national math and science curricula. This last 
mentioned method of looking for statistical similarities is close to the 
method described in this article. The difference is in the nature of the 
data used: the curriculum analyses dealt with the intended curriculum 
whereas the emphasis in this article is on the achieved curriculum, i.e., 
what was actually learned by the students in the countries. 

Conceptual Framework 

Figure 1 presents the difficulty levels (p-values) of 20 items 
forming a part of the TIMSS mathematics test for three countries X, Y, 
and Z. These items have been ordered by their difficulty; that is, the 
actual percentage of students obtaining the right answer for each item. 




x: Y - z 

Figure 1. Item difficulties for 20 TIMSS items for countries X, Y 

and Z. 

It can be seen that the students in countries X and Y performed 
this part of the test relatively similarly, despite the fact that country X 
had higher overall achievement than country Y. The students in these 
two countries performed in a similar way on the same items relative to 
other items. Country Z students performed in quite a different way: the 
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overall students scores on this set of items were about the same as in 
Country X, but the relative national item difficulties for these two 
countries were quite different. This fact becomes more evident if the 
items are re-ordered (see Figure 2) according their difficulty for country 
X (forming "a river" for these two countries). 




X Y Z 

Figure 2. Item difficulties re-ordered for 20 TIMSS items for 
countries X, Y and Z. 

What is the meaning behind such a definition of similarity? The 
simplest explanation is that two countries were simitar if the students in 
these countries performed on the TIMSS math test in a similar way, i.e., 
the same set of items were relatively more difficult or more easy. 

The value of such comparisons of the countries lies in the 
identification of pairs or groups of similar countries. Some of these pairs 
and groups are well-known while some of them have remained 
somewhat hidden. If there is similarity between two countries, the 
question of the cause of such a phenomenon naturally arises; perhaps 
the similarity is the result of a more general relationship between 
countries based on common historical or linguistic similarities, or it may 
be the result of the impact of educational reforms undertaken, or it may 
be just a random occurrence. A brief look at the groups of similar 
countries and an examination of the matching pairs does not support the 
random character of such groupings. Therefore, the naming of similar 
countries, on the one hand, and countries performing in the quite 
different ways, on the other hand, is a contribution not only to general 
knowledge, but can also lead to the objective comparison of the 
curricula of the particular countries, and provide new opportunities for 
educational systems to learn from each other. 

Method 

The TIMSS mathematics test contained a total of 157 items. The 
initial information consisted of the difficulty levels (p-values) of every 
item in every country, i.e., the percentage of the correct answers for 
every item in every country. The range of values for each item was from 
0 to 100 (percent) and could be treated as the countries profile in 
mathematics achievement (measured 157 times, forming a 157- 
component vector). The simple correlation of these values could be 
calculated between all pairs of countries. However, this would not 
produce the desired result. The overall international item difficulty 
would still play a primary role in producing correlation among countries 
(instead of indicating some specific trends), and the correlations 
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between the countries would be high: all countries performed better on 
easy items than on difficult ones. Therefore, the effect of the item 
difficulty should be removed. This can be done in two ways. The first 
would involve a simple ranking of the participating countries from 1 to 
41 according to their results on each item, and the vector of such ranks 
could be defined as a country's profile. This procedure would increase 
the distinguishing feature of the variable (the distance between two 
country's results on the item would be at least one), would take into 
account the results of all TIMSS countries at the same time (the "room" 
for the top 10 is fixed: if one country enters the top 10, another must 
exit), and would not be linked to the overall difficulties of the items. 
These ranks are usually easy to explain, but this artificial numbering 
(the minimal difference " 1 " in ranking might correspond to either a very 
small difference in math achievement results or a very large one) would 
result in a loss of information. Therefore, another method was used: the 
difficulty levels (national p-values) for the item were standardised 
within countries and converted into z-scores. This allowed the TIMSS 
countries to be ranked not by the natural numbers 1, 2, 3 etc., but by 
putting them on a continuous interval scale defined by the countries 
achievement on the item. This procedure removes the effect of the item 
difficulty. In order to avoid the effect of the overall countries 
achievement on the test, the Pearson correlation coefficient for these 
standardized difficulties of the items was calculated to measure the 
similarity between the countries. 

After an examination of the matrix of Pearson correlation 
coefficients of the countries math profiles (i.e., the standardized items p- 
vaiues for countries), a hierarchical cluster analysis was undertaken. The 
average linkage within groups method was used. This procedure was 
used to identify groups of countries having similar relationships in the 
achievement profiles. The reliability of this grouping was based on the 
internal consistency of the group measured by Cronbach's alpha 
coefficient. A cut-off point of 0.75 (of the coefficient alpha) was taken 
for forming a group of countries. The resultant group of countries was 
named and factor analysis was used later in order to extract the factors 
which were the specific profile factors of the group. This two- step 
procedure with the grouping of similar countries first and the factor 
analysis second was selected instead of the straightforward use of factor 
analysis. This was done because the direct factor analysis would have 
taken into account the effect of a main factor in both directions, positive 
and negative. This would have led to the grouping of countries with 
strong opposite trends, and would result in it being quite difficult to 
measure the similarity later. The final analysis of the data consisted of 
an analysis of the similarities between the countries and between the 
groups of the countries. 

One indicator of the reliability of the method could be the 
comparison of the trends of the same country calculated for two 
different grades. The TIMSS study collected data about the students' 
achievement in two adjacent grades called "lower" and "upper" (and for 
most countries these were grades 7 and 8). The same procedure for the 
standardization of the mathematics item difficulties was undertaken for 
both data sets separately and the Pearson correlation coefficients were 
calculated for every TIMSS country pairing for both grades within a 
country. This analysis revealed a very high similarity of profiles for 
these two grades within countries. For mathematics, the coefficients 
were mostly above 0.80. The highest similarities were found for 
Singapore (Pearson correlation coefficient, 0.95), Philippines (0.93), the 



httn ://enaa.asu. edu/ eoaa/v9n3 3/ 



176 



10/29/01 




EPAA Vol. 9 No. 33 Za...: Mathematics and Science Achievement <br>of Various Nation Page 5 of 17 

USA (0.91), Australia (0.92), Korea (0.92), Scotland (0.92), and the 
lowest ones for Bulgaria (0.65), Austria (0.75), Greece (0.76), Spain 
(0.78). For science, these correlations were, for the most part, even 
higher. These results provide evidence of the high stability of country 
trends and, at the same time, could also be regarded as evidence for the 
stability of the method. 

Mathematics 

Analysis of the groups 

Among all 41 countries having taken the TIMSS mathematics test 
for the upper grade, the highest similarity of the mathematics 
achievement profiles was found for England and Scotland. The Pearson 
correlation coefficient for this pair was 0.90. Quite close to these two 
countries were New Zealand (correlation with Scotland 0.86, with 
England 0.85), and Australia (correlations with every of these three 
coimtries above 0.70). Therefore, these countries could be taken to form 
the core of the first group called English-speaking countries. Using this 
language designation, Canada, the USA, and Ireland could also be 
included in this group. The USA correlated most highly with Canada 
but these two North American countries also had much in common with 
the European- Australian group. The place of Ireland was questionable: 
it was more highly correlated with Canada and Scotland. The reliability 
analysis had about the same alpha whether Ireland was included or 
removed from the group. Therefore, all seven English-speaking 
countries were left in the group. An analysis of the matrix of 
correlations presented below showed that Ireland looked like a bridge 
between the European-Australian group and the American one. Ireland 
was more correlated with Canada than with neighboring England. The 
reliability coefficient of 0.87 was quite high and showed some 
homogeneity of the group, despite the fact that the USA was a little to 
the side. 



Table 1 

Correlations of Nations on TIMSS Mathematics Items 
English-Speaking Group 



Australia 


AUS 












alpha 


0.87 


Canada 


CAN 


0.44 


* ★ ♦ 












England 


ENG 


0.76 


0.34 












Ireland 


IRE 


0.37 


0.39 


0.26 


*** 








N.Zealand 


NZL 


0.79 


0.43 


0.85 


0.32 








Scotland 


SCO 


0.73 


0.34 


0.90 


0.35 


0.86 






USA 


USA 


0.30 


0.65 


0.22 


0.35 


0.31 


0.25 


*** 




AUS 


CAN 


ENG 


IRE 


NZL 


SCO 


USA 



Factor Analysis of Nations Correlations 
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communality 


fac 1 


fac 2 


AUS 


0.78 


0.83 


0.30 


CAN 


0.76 


0.23 


0.84 


ENG 


0.91 


0.94 


0.13 


IRE 


0.43 


0.24 


0.61 


NZL 


0.89 


0.91 


0.26 


SCO 


0.89 


0.93 


0.18 


USA 


0.76 


0.07 


0.87 




f_UK 


f_Am 



There were no further TIMSS countries the inclusion of which 
increased the homogeneity of this group as measured by Cronbach's 
alpha. The factor analysis for these seven countries resulted in two 
factors with eigenvalues of 4.06 and 1.35, together explaining about 
77.4 percent of the variation. After rotating these factors (the varimax 
method was used), and examining the loadings, it was easy to find the 
"right" names for them: the first could be called English-speaking 
countries -United Kingdom (f_UK), the second - English-speaking 
countries— America (f_Am). 



Table 2 

Correlations of Nations--Post-Communist Group 



Bulgaria 


BGR 


*** 
















Czech R. 


CZE 


0.07 












alpha 


0.81 




Hungary 


HUN 


0.13 


0.35 














Latvia 


LVA 


0.16 


0.23 


0.13 


*** 










Lithuania 


LTU 


0.24 


0.45 


0.19 


0.59 










Romania 


ROM 


0.41 


0.19 


0.05 


0.31 


0.25 








Russia 


RUS 


0.28 


0.33 


0.20 


0.53 


0.60 


0.52 






Slovakia 


SVK 


0.13 


0.67 


0.39 


0.36 


0.49 


0.36 


0.45 






Slovenia 


SLV 


0.09 


0.29 


0.17 


0.32 


0.34 


0.27 


0.36 


0.44 


* ** 




BGR 


CZE 


HUN 


LVA 


LTU 


ROM 


RUS 


SVK 


SLV 



The second set of candidates to form a group emerged clearly: all 
post- communist countries — Bulgaria, Czech Republic, Hungary, 
Latvia, Lithuania, Romania, Russian Federation, Slovak Republic and 
Slovenia (Central and Eastern Europe CEE countries) — had quite 
similar patterns in their math achievement profiles. An analysis of the 
correlation matrix showed about the same picture as for the English- 
speaking countries in that the place for Hungary was similar to Ireland. 
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Table 3 

Factor Analysis of Post-Communist Group 





communality 


fac 1 


fac 2 


BGR 


0.44 


-0.11 


0.65 


CZE 


0.68 


0.82 


0.09 


HUN 


0.41 


0.64 


-0.07 


LVA 


0.49 


0.34 


0.62 


LTU 


0.59 


0.54 


0.55 


ROM 


0.59 


0.07 


0.77 


RUS 


0.68 


0.38 


0.74 


SVK 


0.74 


0.81 


0.29 


SLV 


0.35 


0.49 


0.33 




f_CE 


f_EE 



The decision to include Hungary in this group was based on the 
same consideration as for Ireland: it had no effect on Cronbach's alpha; 
and, it made sense from a geographical point of view and also allowed 
the extraction of two factors later. 

The factor analysis for this group of these nine countries yielded 
two eigenvalues above one: 3.67 and 1.30, explaining 55.2 percent of 
the total variance. The analysis of the loadings of the factors was not so 
simple as in the previous case: the impacts from the countries were 
somewhat mixed. The first factor was more linked to the Czech 
Republic, Hungary, and the Slovak Republic, and the second to 
Bulgaria, Romania, and the Russian Federation. Countries such as 
Lithuania, Latvia and Slovenia were somewhere in between. Therefore, 
the names Central Europe countries (f_CE), and Eastern Europe 
countries (f_EE) were chosen for these two factors. 

When examining the other TIMSS countries there were no further 
large groups of countries like the two above. Searching for some smaller 
groups one could think of a Nordic group (Denmark, Iceland, Norway, 
Sweden), an East Asian group (Hong Kong, Japan, Korea, Singapore), 

. and possibly about some group from Western Europe such as Austria, 
Belgium, Germany, Greece, France, Spain or Portugal. Unfortunately, 
this last set of countries proved to be very heterogeneous: it was 
possible to identify only pairs of countries (like Austria—Germany, 
French speaking part of Belgium— France), but not more. The Nordic 
countries were quite homogenous (see the table of correlations below). 

Table 4 

Correlations of Nations— Nordic Group 



Denmark 


DEN 






alpha 


0.82 




DEN 


0.49 


0.70 


Iceland 


ICE 


0.46 








ICE 


0.67 


0.82 


Norway 


NOR 


0.47 


0.55 






NOR 


0.72 


0.85 


Sweden 


SWE 


0.41 


0.62 


0.68 


*** 




SWE 


0.73 


0.85 




DEN 


ICE 


NOR 


SWE 




communality 


factor 
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It was possible to identify two other countries that were near to 
the four Nordic countries, namely the Netherlands and Switzerland. By 
adding them to the group there was no increase in the alpha coefficient 
for the group but both of them were quite highly correlated with the 
Nordic group as well as with the UK part from the English-speaking 
group. These two countries form another bridge between the Nordic and 
English-speaking - UK groups and hence the Netherlands and 
Switzerland were not included in either of these groups. The factor 
analysis of the Nordic group yielded one large eigenvalue of 2.6, giving 
one factor, and explaining about 65 percent of variation. 

The Eastern Asian group, formed by Hong Kong, Japan, Korea 
and Singapore, was homogeneous, and each of these four countries 
contributed about the same amount to the Asian factor (one large 
eigenvalue of 2.4 explaining about 60 percent of the variation). 

Table 5 

Correlations and Factor Analysis of Nations—Eastern Asian 

Group 



Hong Kong 


HNK 


*** 




alpha 0.82 




HNK 


0.71 


0.84 


Japan 


JAP 


0.54 


*** 






JAP 


0.63 


0.79 


Korea 


KOR 


0.37 


0.48 


* ** 




KOR 


0.46 


0.68 


Singapore 


SIN 


0.64 


0.42 


0.33 


** * 




SIN 


0.61 


0.78 




HNK 


JAP 


KOR 


SIN 




communality 


factor 



Each of these four countries performed very well on the TIMSS 
test (Singapore was ranked first in overall math achievement, followed 
by the other three). It is interesting to note that there was one country — 
South Africa — with the lowest score of all countries that had a similar 
profile to the Asian group. The math achievement profile of South 
Africa was quite similar to Singapore (correlation coefficient 0.48), to 
Hong Kong (0.41), and Japan (0.34). The inclusion of South Africa in 
this group would not affect the Cronbach's alpha coefficient of the 
group. Considering both the geographical location of these countries 
and the statistical considerations, only four Asian countries were left to 
form this group. From another point of view, this similarity could be 
somewhat artificial because of the method used to measure the 
similarity of pairs of countries with stable trends. In other words, a 
country that performed extremely well on the test could have a similar 
profile to a country that performed extremely poorly on the test because 
on most items country ranks had no diversity; they were always on the 
top, or always on the bottom of the international list. Despite this, it can 
be seen that the method worked well for "average" countries. There 
were no TIMSS countries performing excellently or poorly on every test 
item, and, therefore, the method worked reasonably well to measure the 
similarity of the top (bottom) ranked countries taking into account the 
patterns of the similarity rather the overall achievement results. 

The four groups above were defined according the students' 
achievement on more than 150 math items. These items represented 
different areas in mathematics: fractions, algebra, geometry, and the 
like. The countries traditions in mathematics education placed unequal 
emphasis on these subtopics in the curriculum, and as a consequence of 
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this the students' achievements were also quite different. The similarities 
of profiles for sub-scales are briefly commented on below. 

English-speaking countries . These countries were more similar 
on Proportionality (Cronbach's alpha coefficient is 0.91) than on 
Measurement (0.73). It should be pointed out that there was high 
similarity between the USA and Ireland on Proportionality compared 
with the rest of the countries of this group (USA - New Zealand 0.69, 
Ireland - Scotland 0.80), and the differences on Measurement (the pair 
Australia - Ireland had a negative correlation of -0.3 1). The most 
interesting figures from the other subscales were the 0.64 and 0.69 for 
the pair USA and England on Geometry and Data representation 
respectively (compared with 0.10 for Algebra). 

CEE countries. The analysis of the subscales pointed to the 
exceptional place of Hungary in this group having only negative 
correlations with the other countries in this group on Geometry (up to - 
0.62 with Romania!). Some negative correlations were found in other 
subscales, such as in Fractions Bulgaria - Czech Republic -0.20, in 
Algebra Czech Republic - Latvia -0. 1 6, in Measurement Latvia - 
Romania -0.25, in Proportionality Lithuania - Romania -0.28. 

Generally, the Cronbach alpha coefficients for this group ranged from 
0.55 in Geometry (the Hungarian effect), to 0.85 in Data representation. 
In Geometry Hungary was most similar to the English-speaking 
countries group (correlation with England 0.63, with New Zealand 0.7 1 , 
with USA 0.45). 

Nordic countries. The group is quite homogenious in all 
subscales: there were no negative correlations. The Cronbach alpha 
coefficients ranged from 0.71 {Measurement), to 0.87 {Proportionality). 

East Asian countries. The interesting subscale for this group was 
Proportionality, where the alpha coefficient fell to 0.38 (the correlation 
for the pair Singapore - Hong Kong was 0.71, and with all the rest it 
was about zero). In this subscale Japan was more similar to the 
Philippines (0.79), Korea - to Greece (0.74) or USA (0.68). This 
subscale was one of the shortest with 1 2 items only which might 
account for the diversity. Other subscales worked in a relatively stable 
manner with the alpha coefficients ranging from 0.67 {Algebra) to 0.86 
{Geometry, Measurement). 

Another noteworthy finding from the subscale analysis was 
unexpected and concerns the difference between Measurement and 
Proportionality. Both of these subscales included more application type 
items involving use in real-life situations. Thus, it was expected that 
countries would teach them in a fairly similar way, whereas larger 
differences would be found in topics such as Algebra, Geometry or Data 
representation. 

To summarize, four groups of similar countries were identified 
among 41 TIMSS countries. The relationship between these four groups 
are presented below. Since rotation was used in the factor-analysis, at 
least two pairs of factors in the first two groups should be perpendicular. 
In order to understand the overall picture for all four groups together, an 
examination of the correlation matrix below should be made. 

Table 6 

Correlations Among the Factors 
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f_UK 














f_Am 


0.00 


* ** 










f_CE 


-0.32 


-0.07 


*** 








f_EE 


-0.49 


-0.44 


0.00 


*** 






f_Nord 


0.50 


0.10 


-0.20 


-0.52 


*** 




f_Asia 


-0.25 


0.21 


-0.32 


0.24 


-0.44 


*** 




f_UK 


f_Am 


f_CE 


f_EE 


f_Nord 


f_Asia 



The Nordic group countries were quite similar to the UK group 
(correlation coefficient 0.50), but not so similar to the USA and Canada 
group (0.10), and were opposite to the Eastern European (-0.52) and 
Asian groups (-0.44). Note the Asian group's relationships with 
UK/USA and Central/Eastern European groups. In both cases the 
correlation coefficients were positive with one subgroup and negative 
with another. Thus the behavior of the Asian group could serve as an 
indication that the splitting these two large groups into parts was done 
correctly. 

Science 

Analysis of the groups 



Table 7 

Factor Analysis for Science Items 
English-Speaking Group 





communality 


fac 1 


fac 2 


AUS 


0.64 


0.48 


0.64 


CAN 


0.72 


0.22 


0.82 


ENG 


0.81 


0.88 


0.19 


IRE 


0.61 


0.76 


0.17 


NZL 


0.66 


0.64 


0.50 


SCO 


0.80 


0.87 


0.19 


USA 


0.75 


0.09 


0.86 




f_UK 


f_Am 



The same four groups of countries for mathematics were also 
tested for Science. Two of them were stable enough to be kept for 
science (English speaking countries' group - Australia, Canada, 
England, Ireland, New Zealand, Scotland, USA, and the Nordic group— 
Denmark, Iceland, Norway, Sweden). Relationships in the other two 
groups were more complex and not strong enough to permit keeping the 
countries together, i.e., the CEE countries group (Bulgaria, Czech 
Republic, Hungary, Latvia, Lithuania, Romania, Russian Federation, 
Slovak Republic, Slovenia), and the Eastern Asian group (Hong Kong, 
Japan, Korea, Singapore). One reason for this could be some 
heterogeneity in the school subject called science: in the CEE countries 
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groups, science is taught as separate courses {Geography, Biology, 
Chemistry, Physics). Therefore, the similarities found for one subject 
were not valid for another subject. 

Table 8 

Correlations for Science 
English-Speaking Group 



Australia 


AUS 












alpha 


0.86 


Canada 


CAN 


0.48 














England 


ENG 


0.53 


0.38 


*** 










Ireland 


IRE 


0.34 


0.31 


0.62 


*** 








N.Zealand 


NZL 


0.67 


0.51 


0.59 


0.44 


*** 






Scotland 


SCO 


0.50 


0.38 


0.74 


0.60 


0.59 


*** 




USA 


USA 


0.49 


0.59 


0.25 


0.35 


0.37 


0.27 


*** 




AUS 


CAN 


ENG 


IRE 


NZL 


SCO 


USA 



The link England-Scotland (correlation coefficient 0.74) 
remained the strongest in all TIMSS countries (with an increase up to 
0.82 for Life science, and a decrease to 0.56 for Physics). This group 
was more stable for science than for mathematics: Ireland had more 
weight, and the link USA-Australia was also strengthened. As a 
consequence, the group became more homogeneous, especially in Life 
science where the alpha coefficient was 0.89. Physics was a weaker area 
for this group, but still the similarities between the countries were strong 
enough to keep the group together (the alpha was 0.79). 

The factor analysis for these 7 countries yielded two factors with 
eigenvalues of 3.88 and 1.12, explaining 71.4 percent of the variation. 
After rotation (varimax method was used), an analysis of the loadings 
yielded the names English-speaking countries - United Kingdom (f_UK) 
for the first factor, and English- speaking countries - America (f_Am) 
for the second, as was found in the mathematics achievement results. 

The four Nordic countries of Iceland, Denmark, Norway and 
Sweden formed the second group. The links between three countries in 
this group were about the same for all science sub scales with the 
exception of Environment and nature of science (the alpha was only 
0.40), where Iceland stood out (all correlations with the three other 
Nordic countries were negative!). This country was the weakest member 
of the group in Earth science and in Chemistry, where quite strong links 
to New Zealand (0.64), and the Netherlands (0.58) were found. The 
factor analysis for these 4 countries yielded an eigenvalue of 2.16 
explaining about 54 percent of the variation in the Nordics group. 

Table 9 

Correlations and Factor Analysis for Science 
Nordic Group 
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Denmark 


DEN 


*** 




alpha 0.71 




DEN 


0.60 


0.77 


Iceland 


ICE 


0.36 








ICE 


0.33 


0.58 


Norway 


NOR 


0.43 


0.31 


*** 




NOR 


0.63 


0.80 


Sweden 


SWE 


0.46 


0.20 


0.54 


** * 




SWE 


0.59 


0.77 




DEN 


ICE 


NOR 


SWE 




communality 


factor 



As was mentioned earlier, the other two groups of countries had 
scattered patterns. The East Asian group with Hong Kong, Japan, Korea 
and Singapore split into two pairs: Hong Kong - Singapore (on the total 
science scale the correlation was 0.26, in Physics 0.55, and in Chemistry 
-0.23!), and Japan - Korea (0.36, in Chemistry 0.66, compared with - 
0. 1 1 for Earth science). The Cronbach alpha for the total science scale 
was 0.47, but it ranged from -0.08 for Environment and Nature of 
science to 0.57 for Life science and 0.53 for Chemistry. Nevertheless, 
these figures were too low to coalesce the countries into one group. 

The eight CEE countries formed a group in mathematics, but for 
science the situation was more complex. 

Table 10 

Correlations for Science 
Post-Communist Group 



Bulgaria 


BGR 


*** 
















Czech R. 


CZE 


0.07 


*** 








alpha 0.71 




Hungary 


HUN 


0.04 


0.18 


*** 












Latvia 


LVA 


0.14 


0.00 


0.27 


* ** 










Lithuania 


LTU 


0.18 


0.24 


0.15 


0.39 


*** 








Romania 


ROM 


0.29 


0.34 


0.21 


0.03 


0.30 








Russia 


RUS 


0.24 


0.24 


0.12 


0.32 


0.48 


0.44 






Slovak 

R. 


SVK 


0.05 


0.51 


0.18 


0.01 


0.09 


0.28 


0.22 


*** 




Slovenia 


SLV 


0.03 


0.32 


0.22 


0.25 


0.16 


0.33 


0.12 


0.31 


** * 




BGR 


CZE 


HUN 


LVA 


LTU 


ROM 


RUS 


SVK 


SLV 



Formally, the measure of the homogeneity of the group - 
Cronbach's alpha for overall science scale - was quite high and was 0.71 
(ranging from 0.68 for Physics, to 0.78 for Chemistry). A cut-off point 
of 0.75 was defined for alpha to form a group. For the Nordics group 
and for the CEE group, the coefficients were somewhat lower than the 
cut-off point - 0.71 in both cases. The Nordic countries were grouped, 
but the analysis of the matrix of the correlations for CEE countries 
group showed large diversity among countries, especially in the 
relationships for the different subjects of science. Firstly, Bulgaria was 
different: in Physics this country was similar to the Russian Federation 
(0.41) and Romania (0.41), but in Chemistry similar to Japan (0.49). 

The subscale Environment and Nature of science disturbed this 
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grouping. Also noteworthy are the links between the pairs Hungary - 
Germany (0.79), Lithuania - France (0.48), Russia - Greece (0.59), 
Slovakia - Greece (0.65) as well as Lithuania- Slovenia (0.66), Russia - 
Slovakia (0.74). The factor analysis yielded three eigenvalues above 1, 
and the explanations of these three factors became quite complex. 
Therefore, the decision was made not to form the CEE countries into 
one group. 

Discussion 

The process of the grouping of countries indicated the high 
similarity between some countries, but from another point of view 
showed that some countries were different from these groups. An 
examination of some of the countries not included in the groups has 
been undertaken. The relationship of these countries with the main 
trends of groups, extracted by factor analysis above, is mentioned first, 
followed by some analyses of the pairing of countries. 

Only two significant links with factors could be seen for Austria: 
this was in mathematics with Central Europe countries f_CE 
(correlation 0.36), and with East Europe countries f_EE. (-0.23). Austria 
was most linked to its neighbor Germany (0.40 in math, 0.58 in science, 
with 0.85 for Life science'). Other interesting links were with Norway 
(0.51 in Algebra), Switzerland (0.41 in science), Czech Republic (0.49 
in Life science), Ireland (0.55 in Chemistry), Israel (0.52 in Chemistry). 
For Earth science some obscure relationships could be noted with 
Kuwait (0.55), and the Philippines (0.44). 

There were no significant correlations for France with other 
countries, except a weak 0.22 in mathematics with Nordic countries 
f_No. It was not unexpected that France in mathematics was most 
similar to the French part of Belgium (Belgium took the part in TIMSS 
education with two subsystems - Flemish and French, according to the 
prevalent language used in education in the country). The correlation 
coefficient was 0.33. Some of the CEE countries were traditionally 
influenced by French ideas in mathematics education and this fact can 
be seen in the analyses: in the suhscalsAlgebra, France was correlated 
with the Russian Federation (0.32), in Geometry with the Czech 
Republic (0.49), and Lithuania (0.47). The French part of Belgium was 
most similar to France in Science (0.67, with an increase to 0.87 in 
Chemistry). Some other interesting links were in Physics to Denmark 
(0.45) and in Chemistry to Switzerland (0.69). 

The most interesting links for Germany were the negative 
relationships with the East Asian factor f_As (-0.5 1) and with East 
Europe factor f_EE (-0.32). At the same time, Germany was quite 
similar to the United Kingdom part of the English-spe^ing countries 
group (correlation with f_UK was 0.41). Germany, as was mentioned 
above, was similar to Austria. In math some other links to Switzerland 
(0.41), Norway (0.40), and in Algebra to Greece (0.67), Portugal (0.56) 
could be mentioned too. Switzerland was similar to Germany in science 
(0.54), together with France (0.50), Norway (0.49) in Chemistry and to 
the Netherlands in Physics (0.42). 

There were threecorrelations with factors for Israel: two of them 
negative (-0.22 with f_UK, and -0.30 with f_No), and one positive 0.21 
with f_EE. The country most similar to Israel in mathematics was 
Cyprus (0.36), especially in Algebra (0.60), and Geometry (0.53). There 
was also a high correlation between Israel and the Russian Federation in 
Geometry (0.55). It was difficult to identify similarities with Israel in 
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science, except some links with Cyprus (0.44 in Earth science), and 
Austria (0.52 in Chemistry). 

The Netherlands was like anintemational cross-roads: in 
mathematics +0.62 with f_UK, -0.48 with f_EE, +0.45 with f_No, - 

0.32 with f_CE. For science the relationships were weaker: 0.23 with 
f_Am, and 0.24 with f_No. In mathematics there were common patterns 
with Austria (0.52, with an increase to 0.66 in Algebra), England (0.56), 
New Zealand (0.60), Sweden (0.59), and in Geometry to the USA 
(0.64). The neighbouring education system, Belgium-Flemish, was most 
similar in science (0.48, with increase till 0.57 in Physics, and even to 
0.82 in Chemistry). Some other interesting links were: in Physics with 
Germany (0.42), in Chemistry with Switzerland (0.64), in Earth science 
with Sweden (0.51). 

Spain had something in common in mathematics with f_CE 
(0.26), but was negatively correlated with Nordics and East Asian 
countries (-0.22 with f_No, -0.28 with f_As). Spainin mathematics was 
most similar to its neighbor Portugal (0.39, with an increase to 0.58 in 
Geometry), with a possible link to Slovenia (0.37), and some 
relationship with the USA (0.46 in Geometry). The linguistic similarity 
had an impact on science education: correlation with Colombia (0.27), 
Portugal (0.39, with an increase to 0.61 in Earth science). Some other 
similarities were with the USA (0.46 in Physics), Greece (0.62 in 
Chemistry), and Norway (0.59 in Earth science). 

In mathematics the pattern of Switzerland was interesting (+0.49 
with f_No, -0.41 with f_EE, +0.29 with f_UK, -0.28 with f_As), and in 
science there was just one significant link (0.32 with f_No). Switzerland 
was quite similar in mathematics to Nordic countries Norway (0.37) and 
Sweden (0.46), and had common patterns in Geometry with New 
Zealand (0.58) and Canada (0.44). In science Switzerland was similar to 
Belgium-French (0.41, with an increase to 0.67 for Physics, 0.78 for 
Chemistry), as well as was linked with Austria (0.51 in Life science), 
France (in Physics 0.43, in Chemistry 0.69), the Netherlands (in 
Chemistry 0.64), Germany (in Earth science 0.63), Belgium- Flemish 
(in Chemistry 0.81). 

This article could be viewed as an attempt to measure something 
that some might say was unmeasurable. It is impossible to measure 
* everything in education by converting the complexities and 
idiosyncrasies of nations into figures. Tables with correlation 
coefficients presented in this paper do not explain everything that is 
going in the schools of these countries. But sometimes it is useful to see 
"well-known truths" converted into the language of figures. 

Notes 

1 . Many visions, many aims. A cross-national investigation of 
curricular intentions in school mathematics. Kluwer, 1 996. 

2. Many visions, many aims. A cross-national investigation of 
curricular intentions in school science. Kluwer, 1996. 

3. Mathematics Achievement in the Middle School Years: lEA 's 
Third Intentional Mathematics and Science Study (TIMSS). 

Boston College, 1996. 

4. Science Achievement in the Middle School Years: lEA 's Third 
Intentional Mathematics and Science Study (TIMSS). Boston 
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Abstract 

Although international comparisons of average student 
performance are a staple of U.S. educational debate, little 
attention has been paid to cross-national differences in the 
variability of performance. It is often assumed that the 
performance of U.S. students is unusually variable or that 
the distribution of U.S. scores is left-skewed - that is, that 
it has an unusually long ‘tail’ of low-scoring students - but 
data from international studies are rarely brought to bear 
on these questions. This study used data from the Third 
International Mathematics and Science Study (TIMSS) to 
compare the variability of performance in the U.S., 
Australia, France, Germany, Hong Kong, Korea, and 
Japan; investigate how this performance variation is 
distributed within and between classrooms; and explore 
how well background variables predict performance at 
both levels. TIMSS shows that the U.S. is not anomalous 
in terms of the amount, distribution, or prediction of 
performance variation. Nonetheless, some striking 
differences appear between countries that are potentially 
important for both research and policy. In the U.S., 
Germany, Hong Kong, and Australia, between 42 and 47 
percent of score variance was between classrooms. At the 
other extreme, Japan and Korea both had less than 10 
percent of score variance between classrooms. Two-level 
models (student and classroom) were used to explore the 
prediction of performance by social background variables 
in four of these countries (the U.S., Hong Kong, France, 
and Korea). The final models included only a few 
variables; TIMSS lacked some important background 
variables, such as income, and other variables were 
dropped either because of problems revealed by 
exploratory data analysis or because of a lack of 
significance in the models. In all four countries, these 
sparse models predicted most of the between-classroom 
score variance (from 59 to 94 percent) but very little of 
the within-classroom variance. Korea was the only 
coxmtry in which the models predicted more than 5 
percent of the within-classroom variance in scores. In the 
U.S. and Hong Kong, the models predicted about one- 
third of the total score variance, and almost all of this 
prediction was attributable to between-classroom 
differences in background variables. In Korea, only 19 
percent of total score variance was predicted by the 
model, and most of this most of this was attributable to 
within-classroom variables. Thus, in some instances, 
countries differ more in terms of the structure and 
prediction of performance variance than in the simple 
amount of variance. TIMSS does not provide a clear 
explanation of these differences, but this paper suggests 
hypotheses that warrant further investigation. 
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Introduction 

International comparisons of average student performance are widely 
discussed by policymakers and the press and have had a powerful influence on 
educational debate and policy in the US. In an era when traditional norm- 
referenced reporting of student performance ostensibly has gone out of favor, 
"country norms" have become an increasingly important indicator of the success 
of US education and the levels of performance to which this country should 
aspire. The publication of the results of the Third International Mathematics and 
Science Study (TIMSS) over the past several years (Beaton et al., 1996a, 1996b; 
Mullis et al., 1997, 1998) has increased further the prominence of international 
comparisons in the US debate. 

Much of the discussion of international comparisons has focused on horse- 
race comparisons of means or medians. Although presented in TIMSS reports, 
information on the variability of student performance has usually been ignored in 
the US debate or has been used in a lopsided and potentially misleading fashion. 
Typically, the variability in the US has been considered, while the variability in 
the countries to which the US is compared has been ignored. For example, 
earlier this decade, the results of the 1991 International Assessment of 
Educational Progress (lAEP) were projected onto the National Assessment of 
Educational Progress (NAEP) scale, permitting comparison of countries 
participating in lAEP to states participating in the 1992 NAEP Trial State 
Assessment in mathematics. These comparisons, which have been widely cited, 
showed that the highest-scoring US states, such as Iowa and North Dakota, had 
mean scores similar to those of the highest-scoring countries, such as Taiwan and 
Korea (National Center for Education Statistics, 1996, Figure 25). High-scoring 
regions in Taiwan and Korea, however, were not compared to the US mean. 

Underlying some of these comparisons appears to be an expectation that 
the variability of student performance is atypically large in the US. Indeed, some 
observers have made this expectation explicit. For example, Berliner and 
Biddle, in disparaging the utility of international comparisons of mean 
performance, wrote: 

The achievement of American schools is a lot more variable than is 
student achievement from elsewhere.... To put it baldly, American 
now has some of the finest, highest-achieving schools in the 
world — and some of the most miserable, threatened, underfunded 
educational travesties, which would fail by any achievement 
standard (1995, p. 58, emphasis in the original). 

To buttress this assertion, they cited the NCES comparisons of US states 
and foreign nations noted above, which displayed no information about the 
variation of performance in other countries and included no information about 
the variation of performance among schools within any country. 

Research Questions 

This study was undertaken to explore the variability of performance in the 
US and several other countries using TIMSS data. Specifically, the study 
explored two primary questions: 

1 . How large is the performance variation in our sample countries, and how 
is this variation distributed between and within classrooms? 



httD://eDaa.asu.edu/eDaa/v9n34/ 



192 



10/29/01 




EPA...: Predicting Variations in Mathematics Performance in Four Coimtries Using TIMS Page 4 of 28 



2. How well do background variables predict performance variation in our 
countries, both within and between classrooms? 

The results reported here are limited to mathematics in the higher grade in 
Population 2 (grade 8). We focused on Population 2 rather than Population 1 
(elementary grades) because of doubts about the validity and utility of self-report 
data from elementary school students.(Note 1) Population 3 (end of high school) 
presented formidable difficulties of sample non-equivalence. The analyses 
focused on mathematics because the TIMSS sample design which selected 
students based on the mathematics classes they attended rather than the science 
classes (Foy, Rust, and Shleicher, 1996, p. 4-7). This precluded decomposition of 
score variation and hierarchical modeling in science. 

Methods 

To answer these research questions, our analyses proceeded in two steps: 

1 . We compared the distributions of student-level performance across all the 
countries in the Population 2 sample. 

2. We used a smaller, purposive subsample of countries to analyze the 
variability in student performance between and within classrooms and to 
explore the contributions of student background characteristics to both of 
these sources of variability. 

The performance measure used in all analyses was BIMATSCR, the 
"international mathematics achievement score" (Gonzalez, Smith et al., 1 997) 
used in TIMSS published reports for Population 2. Technically, BIMATSCR is 
not a score in the traditional sense, but it is labeled a score here for simplicity. 
TIMSS was designed to provide aggregate estimates but not scores for individual 
students. In lieu of scores, TIMSS provides for each student five plausible 
values, which are "random draws from the estimated ability distribution of 
students with similar item response patterns and background 
characteristics" (Gonzalez, Smith et al., 1997, p. 5-1). In this respect, TIMSS 
followed a variant of the procedures NAEP has used since 1984. In the case of 
Population 2, however, scores were conditioned only on country, gender, and 
class mean, not on background variables (Gonzalez, 1998). In theory, the 
variance of repeated estimates using different plausible values should be added 
to the sampling variance to obtain an estimate of error variance for statistics 
calculated with plausible values. However, Gonzalez, Smith et al. (1997, p. 5-8) 
report that the intercorrelations among TIMSS plausible values are so high that 
this error component can be ignored. It was not calculated for statistics reported 
in this paper. 

The step 1 analyses are purely descriptive and use data available in TIMSS 
publications (Beaton et al., 1996a and 1996b; Mullis, et al., 1997, Martin et al., 
1997). 

Our initial purposive subsample for the more detailed analyses in step 2 
included seven counties: Australia, France, Germany, Japan, Hong Kong, Korea, 
and the US. Japan and Korea were selected because they are often used as 
examples of high-performing countries in comparisons with the US. Germany 
was included because it is often noted in discussions of the competitiveness of 
the US workforce. Hong Kong was included because it has both parallels with 
and interesting differences from Japan and Korea. France was included because 
in eighth-grade mathematics, it showed an unusually small variemce of 
performance. Australia was considered primarily for methodological reasons. 
Although we present some results for all seven countries, we limited modeling of 
the predictors of variance to four: the US, France, Hong Kong, and Korea. 
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Students in Japan did not complete the survey items used in the modeling. 
Response patterns for students in Germany made us suspicious of that country's 
data. Since Australia was included more for methodological than for substantive 
reasons, we dropped it from the modeling because of similarities in the 
preliminary results from Australia and other countries. 

In our second stage analyses we decomposed the variance among students 
scores from each of the countries into the variance within classrooms and the 
variance between classrooms, and in the four primary countries, we explored the 
predictors of variance at each of these levels. Ideally one would want to 
decompose the variance into at least three levels: within classrooms, between 
classrooms within schools, and between schools. The school and classroom 
levels of aggregation are not exchangeable. For example, a decision to track 
students on the basis of ability would increase the variance between classrooms 
within schools while decreasing the variance within classrooms, but it would not 
directly affect the variance between schools. Conversely, residential segregation 
on the basis of social class would increase performance variance between 
schools, but it could decrease the variance between classrooms within schools by 
making schools more homogeneous with respect to achievement. 

In all countries other than the US, Australia, and Cyprus, however, the 
TIMSS Population 2 sample consisted of a single classroom per school. 
Therefore, in most countries, one can only specify a two-level model in which 
variations in performance between schools and between classrooms within 
schools are completely confounded. Accordingly, we decomposed the 
variability in math scores from each of the four countries into within classroom 
variability and between classroom variability. The between classroom 
variability includes contributions from both the variation of classrooms within 
schools and the variation between schools. 

To fit these models we sacrificed some of the richness of the US data in 
order to obtain comparable to the results from all four countries. We did this by 
(^creating a subsample of the US samples that consisted of a single classroom per 
school, randomly selected from the multiple classrooms in the original sample. 
We modified the sample weights and jackknife replicates used in variance 
estimation accordingly. 

Our step 2 analyses followed the same course in each country and 
extended from simple exploratory data analysis (EDA) to hierarchical modeling. 
Extensive EDA was used to explore individual-level and classroom-level 
variations in performance and background variables, to determine whether 
background variables showed sufficient variability to be usable in analysis, to 
determine whether the relationships between background variables and 
performance appeared sensible, and to decide whether and how to categorize 
variables. The patterns uncovered by this EDA substantially constrained our 
analyses in several instances. 

Simple bivariate relationships between performance and background 
variables were examined for all of the variables considered for the hierarchical 
models. When necessary, variables were recoded so that a positive relationship 
with scores would be expressed as a positive correlation. The bivariate analyses 
were carried out three ways because of the inherently hierarchical nature of the 
data: (1) student-level uncentered (i.e., simple student-level analyses without 
regard to classrooms); (2) student-level, centered on classroom means 
(corresponding to the within-classroom component of variance); and 
(3) classroom-level (corresponding to the between-classroom component of 
variance). 

Hierarchical modeling using multiple background variables followed 
bivariate analyses. The models include the classroom mean for each background 
variable and the individual student-level values, centered on classroom means. 
With centering, the coefficients produced by the model separately measure each 
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variable's contribution to both the between- and within-classroom variability. 

TIMSS used a complex sampling plan with unequal probability of 
selection among schools from each country's sample. To account for this 
disproportionate sampling, all analyses reported here are weighted unless noted. 

Weighted analyses produce consistent estimates of model parameters even if the 
sample design is disproportionate or more technically nonignorable (see, e.g., 

Pfefferman, 1996 for discussion on the use of weights in model fitting). We 
used the methods of Pfefferman et al. (1998) to fit our weighted hierarchical 
models using specially written SAS macros. (For the macro and more detail on 
methods, see Koretz, McCaffrey, and Sullivan, 2000.) 

Distributions of Student-Level performance in TIMSS 

Basic information about the size of the performance variation in 
participating countries, analyzed at the level of students without regard to 
aggregation, is provided in TIMSS publications. Appendices to the reports 
provide standard deviations and selected percentiles (5th, 25th, 50th, 75th, and 
95th) of the performance distributions (Beaton et al., 1996a and 1996b, 

Appendix E; Mullis et al., 1997, Appendix C; Martin et al., 1997, Appendix C). 

At the level of individual students, the eighth-grade mathematics 
performance of US students was near the median of the 3 1 countries that met the 
TIMSS sampling requirements for the eighth grade (see Beaton, et al., 1996a, 

Tables 2.1 and E.3). The country-level standard deviations varied greatly, from 
58 to 110, but half were clustered in the narrow range from 84 to 92. The 
median standard deviation across the 31 countries was 88. The standard 
deviation of the US sample was 91, only slightly above the international 
median. Among these 3 1 countries, the country-level standard deviation of 
eighth-grade mathematics performance was strongly predicted by country means; 
the higher the mean, the larger the standard deviation (r=.71; see Figure 1). Seen 
this way, the standard deviation of mathematics performance in the US was 
about nine percent higher than the value that would be predicted from the US 
mean. Numerous other countries, however, had standard deviations that deviated 
comparably from those predicted by their means. For example, clustered tightly 
eu-ound the US in Figure 1 are England and New Zealand, and Germany would 
be as well if it were included in Figure 1. Germany does not appear in Figure 1 
because it did not meet all sampling requirements. (In eighth-grade science, the 
standard deviation in the US was indeed one of the largest, but it is not an 
outlier; see Koretz, McCaffrey, and Sullivan, 2000.) 




Figure 1. Plot of Mathematics Standard Deviation 
by Mathematics Mean, Grade 8, 

31 Countries Meeting Sampling Requirements 
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(based on Beaton et al., 1996a) 

Figure 1 rebuts the common notion that high-scoring Asian countries have 
a more equitable (i.e., narrower) dispersion of performance, at least in eighth- 
grade mathematics. All three of the Asian countries in our sample have larger 
standard deviations than does the US: Hong Kong's and Japan's standard 
deviations are roughly 10% larger than that in the US, and Korea's is 
approximately 20 % larger. Among our sample of seven countries, only France 
has an unusually small standard deviation of eighth grade mathematics 
performance, either in absolute terms or relative to its mean. 

In grade 8 mathematics, TIMSS also calls into question the view that the 
US mean is pulled downward by a distribution with an unusually long left-hand 
(low-scoring) tail. As shown in Figure 2, the US distribution shows a slight 
right-hand skew rather than a left-hand skew. The US mean is not pulled 
downward because of a small number of low scoring students. Figure 2 
compares the US distribution to the data from Korea. The Korean distribution is 
substantially wider, as its larger standard deviation indicates. The right-hand 
tails of the distributions in the two countries are nearly parallel. The left-hand 
side of the distribution is much shorter in the US, however, pulling the US tail 
closer to the Korean tail. (Note 2) 



<n 

c. 

0 > 
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200 40O 600 800 lOOO; 

Mathematics Score 

Figure 2. Distributions of Mathematics Scores, 
Grade 8, Korea and US. 

This plot is unweighted. Weighting has virtually no effect on 
the distribution of scores in Korea and only a trivial effect on 
the distribution in the US. 




Simple Decomposition of Performance Variance in Four Countries 

The previous discussion demonstrates that the overall distribution of 
student level performance in the US is not anomalous. However, looking only at 
the overall variability might miss important differences between performance in 
the US population compared to that of other countries. For example, the extent 
to which the variability is clustered, e.g., within classrooms or schools, might 
vary across countries. In addition, the possible sources of the variance might 
also differ across countries, which would suggest different interpretations of the 
variability of performance and different policy responses to low mean 
performance in the US. We used data from all seven countries to determine the 
clustering of variability within and between classrooms. As noted above, we 
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focus on the classroom rather than the school because the TIMSS sample makes 
it impossible to distinguish clustering within schools from clustering within 
classrooms. 

The decomposition of mathematics score variance into within- and 
between-classroom components is sufficient to reveal striking differences among 
the seven countries in our sample. In the US, Hong Kong, Germany, and 
Australia, a bit over half of the total variance in eighth-grade mathematics scores 
lies within classrooms (Table 1). In contrast, in Japan and Korea, over 90 
percent of the variance lies within classrooms. France is intermediate, with 
about three-fourths of the total variance lying within classrooms. 

Table 1 

Percent of Score Variance Within and Between Classrooms 



Country 


Percent Between 


Percent Within 


Australia 


47% 


53% 


France 


27 


73 


Germany 


45 


55 


Hong Kong 


46 


54 


Japan 


8 


92 


Korea 


6 


94 


US 


42 


58 



Similarities among some countries in this decomposition of variance, 
however, might mask important differences that would be come apparent if 
TIMSS made it possible to distinguish between-school from between-classroom 
variance. For example, Schmidt, Wolfe, and Kifer (1993) partitioned the 
variance of eighth grade mathematics scores in six countries using data from the 
Second International Mathematics Study, which had two classrooms per school 
in a number of countries. They found striking differences among countries in the 
partitioning of aggregate variance. In France, for example, they found that two- 
thirds of the aggregate variance lay between schools, while in the US, only 9 
percent of the aggregate variance lay between schools (with the remainder lying 
between classrooms within schools). 

The average classrooms in our sample of seven countries differ strikingly 
in their heterogeneity of performance, with the US showing relatively little 
variability within classrooms. The heterogeneity of performance within 
classrooms depends on both the total variance of performance in each nation and 
the breakdown of this variance into within- and between-classroom components. 
Japan and Korea have slightly larger national standard deviations than the US in 
Population 2 mathematics and also have a much larger share of their total 
variance lying within classrooms than does the US. Therefore, the typical 
within-classroom standard deviation in mathematics is considerably larger in 
Japan (96) and Korea (102) than in the US (74). (See Table 2.) The average 
classrooms in France, Germany, Hong Kong, and Australia are more similar to 
that in the US in heterogeneity. 



Table 2 

Within-CIassroom Standard Deviations 
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Country 


Standard Deviation 


Australia 


83 


France 


63 


Germany 


64 


Hong Kong 


73 


Japan 


96 


Korea 


102 


US 


74 



Multilevel Models of Performance Variation 

As noted, we used data from four countries, the US, France, Hong Kong, 
and Korea, to explore the relationships between performance variation and 
background variables. 

Based on research showing which background characteristics predict 
student performance in the US, we chose to examine parental education, other 
measures of socioeconomic status and family composition, measures of 
academic press in the family and community, and a few measures of student 
attitudes. We also examined the effect of student age, which could predict 
performance in at least two ways. Through maturational effects, older students 
might be expected to perform better than others do. On the other hand, to the 
extent that students who do poorly in school are held back in grade, older 
students in a given grade might be expected to perform more poorly than others, 
particularly in the higher grades. Variations in age at entry could also affect later 
scores in several ways. 

We did not examine curricular variables. As measured, these will not 
predict variation within classrooms, and research in the US has generally shown 
variations in schooling to be less powerful predictors of performance than 
background factors. However, curricular differences may be important 
predictors of performance variation between classrooms within schools (for 
example, when students are tracked by ability) and between schools (when 
schools differ substantially in curriculum). Moreover, important curricular 
variables are likely to be correlated with background variables. Thus, the results 
we report here should not be interpreted as clear effects of background variables. 
Rather, they are likely joint effects of the measured background factors, 
educational factors collinear with them, and other omitted variables correlated 
with the measured variables. 

Selecting Variables for Inclusion 

As noted, exploratory data analysis revealed limitations in some variables 
that constrained their use in format models. The few examples presented here 
illustrate that EDA has particular importance in comparative, international 
studies because variables may behave differently in different countries. 

Although TIMSS includes numerous attitude and press variables, we 
focused on a set of 1 5 Likert variables that asked students how strongly they 
disagreed or agree with statements that the student's mother, the student's friends, 
and the student herself considered it important to do well in mathematics, do 
well in the language of the test, do well in sports, be in a high-achieving class, 
and have time to have fun. EDA showed these press and attitude variables to be 
problematic in several respects. In some instances, responses showed little 
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variation. Some relationships with scores were not what one would anticipate if 
the variables were measuring the intended constructs. In several instances, data 
showed suggestions of response bias. 

For example, several problems can be seen in the responses of eighth- 
grade students to the BSBMMIP2 press for achievement variable, "My mother 
thinks it is important for me to do well in mathematics at school" (Figure 3). 
Each of the six panels arrayed across Figure 3 represents the results from a 
different country. In the figure we include the four countries in our analysis 
sample as well as Australia and Germany; this item was not administered in 
Japan. The common vertical axis, labeled BIMATSCR, is the final TIMSS 
mathematics score. The four categories of responses to the survey question are 
arrayed on the X-axis of each panel: SD = strongly disagree, D = disagree, A = 
agree, and SA = strongly agree. The vertical position of each plotted circle 
indicates the mean score of the students in that country who gave that particular 
response to the background question. The radius of each circle is proportional to 
the percent of students within each country who provided that particular 
response. The range of sizes is constrained to make the graphic intelligible, 
however, and in the case of variables with extreme differences in cell counts, 
including some cells in Figure 3, the relative sizes of the circles understate the 
actual differences in cell counts. 




RGSPOMSE 

Figure 3. Mathematics Scores and Responses to BSBMMIP2 Press Variable 

In all the six countries other than Germany, the relationship between 
scores and responses to the "My mother thinks it is important for me to do well 
in mathematics at school" variable was in the anticipated direction: the more 
strongly students agreed with this statement, the higher their average scores. In 
most countries, however, this relationship stemmed in large measure from very 
small groups of students who "disagree" or "strongly disagree" with this 
statement, and the group that included most students showed only weak 
relationships. In the US, for example, 97 percent of all students are in the 
"strongly agree" and "agree" categories, the mean mathematics scores of which 
differed by only 10 points. The "disagree" and "strongly disagree" categories 
had markedly different score means but contained only 2 and 1 percent of 
students, respectively. This variable is likely to have relatively little utility in 
predicting score variability in the sampled countries, even if maternal press for 
achievement is an important influence. 

The data from Germany in Figure 3 show an unusual pattern and 
demonstrate the value of EDA. The relationship between this press variable and 
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scores is not monotonically positive in Germany; the strongly agree and strongly 
disagree groups had approximately the same mean scores. This pattern, which 
appeared repeatedly across the TIMSS press and attitude variables in the German 
data, calls the validity of the responses into question. Because of patterns such 
as these and the less than optimal sampling in Germany, we did not model the 
relationships between background variables and scores in Germany. The 
extremely strong positive relationship in Korea, which also appeared repeatedly, 
was also grounds for concern. For example, the strong very strong positive 
relationship appearing in Korea extended to "I think it is important be placed in 
the high achieving class," even though eighth-grade classes are not tracked by 
achievement in Korea (Hyung Im, 1998). However, the response patterns in 
Korea to the variables we used in modeling were not sufficiently suspect in our 
judgment to warreint excluding Korea from modeling. 

The relationships between some other press variables and student 
performance varied markedly, sometimes dramatically, among countries. These 
differences among countries could have several causes. There might be response 
biases, either consistent or item-specific, that vary among countries. Translation 
problems could engender misleading response differences. There might be 
substantive reasons for these differences as well; for example, press variables 
might in fact have stronger relationships with student performance in some 
countries than in others, perhaps because of differences in the correlations 
between press variables and school characteristics or between press variables and 
ethnicity. 

TIMSS also includes press variables that one would expect to show weak 
or even negative relationships with scores. One set, for example, asks students 
how strongly they agree with the statements that mother, friends, and the student 
herself think it is important to have time to have fun. One might expect that 
students who think it particularly important to save time for fun might be less 
willing to put long hours into study and would therefore score lower. T>wo of 
the strongest positive predictors of mean scores from this set of variables, 
however, are the strength of agreement with the statements "I think it is 
important to have time to have fun" (BSBGSIP4) and "My friends think it is 
important for me to have time to have fun" (BSBGFIP4). 

In response to these findings, we used only two of these 1 5 press variables 
in our models: the strength with which the student agreed that the mother and the 
student herself consider it important to do well in mathematics. We pooled these 
two variables for each subject, creating a single "press for mathematics variable" 
variable from the students' responses pertaining to themselves and their mothers. 
These composites were the mean of the two variables for the subject when both 
were present and whichever was present when one was missing. The decision to 
pool these two variables, which is consistent with the logic of Likert scales, was 
made because the two press variables taken individually had only insubstantial 
relationships with scores, while the composite showed stronger relationships 
with scores. 

We also examined the quality of data for 10 student and family 
background variables:, whether the student was born in the country of testing; 
mother's and father's educational attainment; number of people in the home; 
whether the father, mother, and any grandparents lived with the student; how 
many books were in the home; and whether the home had a study desk and a 
computer. Fewer problems appeared with background variables than with press 
and attitude variables. Missing data and "I don't know" responses, however, 
posed serious difficulties, particularly in France. 

In all of our countries, responses to the questions about parents' 
educational attainment were missing for a substantial percentage of students. 

This problem was particularly severe in France (where 17 percent were missing 
for fathers and 16 percent for mothers). More important, of the students who 
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responded to these question, many answered "I don't know." This problem was 
particularly severe in France, where 34 percent responded "I don't know," so that 
a total of 50 percent of respondents provided no informative answer (Table 3). 

Efforts to impute values were unsuccessful. Thus, we had to choose between 
omitting parental educational attainment from models in France in order to use 
most of the sample, or including parental education and using a substantially 
reduced sample. We opted to include mother's education at the cost of using a 
reduced sample. Comparisons of preliminary models indicated that the choice 
between these options probably affected parameter estimates but did not to have 
a major on the overall prediction of score variance. Although our interpretation 
focuses on the latter, any interpretation of the results from France should be 
taken with caution because of this limitation of the data. 

Table 3 

Percent of Students with No Response or Response of "I don't 

Know" 

to Question About Mother's Educational Attainment 





Missing 


I don't know 


Total 


Australia 


4% 


15% 


18% 


France 


13 


34 


47 


Germany 


9 


21 


30 


Hong Kong 


5 


9 


14 


Korea 


0 


9 


9 


USA 


3 


7 


11 



Two variables, mother's educational attainment and number of books in 
the home, illustrate another issue that can arise in comparative studies - that is, it 
may be desirable or necessary to treat variables differently in different coimtries. 
Both variables showed substantial but not always monotonic positive 
relationships with aehievement. For example, the mean mathematics scores of 
students whose mothers were in the "finished secondeuy" and "some vocational" 
categories were not in the same order in all countries. We combined these 
categories in all countries except Hong Kong. In Hong Kong, small samples and 
a different pattern of means suggested collapsing the "some voeational" category 
of maternal education with "finished university." Similarly, in France only, the 
mean scores of the students reporting the largest number of books was lower 
than that of the category below, so we collapsed those two categories into a 
single category for the French model. We did not collapse these groups for other 
countries. 

The variables used in the final modeling are noted in Appendix A. 

Specifying Multilevel Models 

The multilevel models reported here are simple "fixed coefficients" 
models (Kreft and DeLeeuw, 1998). That is, the coefficients estimating the 
level-one relationships between background factors and achievement (student- 
level relationships within classrooms) are held constant across classrooms wdthin 
countries. Between-classroom effects were thus limited to differences in 
intercepts. In general form, this model is: 

y„ =a + -Xj) + yxj + s,j 

...where the subscript i indicates individuals, y indicates classrooms, an 
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underscore indicates a vector, and a bar over a variable indicates a mean. That 
is, a student's score reflects a vector of background variables weighted by a 
vector of regression coefficients, a vector of classroom means of those same 
background characteristics weighted by a second vector of coefficients, and 
random error. The coefficients applied to individual characteristics are 
unaffected by classroom characteristics. (That is, there eire no cross-level 
interactions.) Equivalently, this can be expressed in terms of two levels as 
follows: 



ytj =aj+/}{x,j-xj) + sl 

aj = a + yxj + rjj 

S = 77 + 5 ** 

V h y 

In other words, the intercept in each classroom is the sum of the overall 
intercept and the sums of the classroom aggregate variables weighted by the 
classroom-level regression coefficients, plus error. The score of each individual 
student is then the sum of that student's classroom intercept and the sum of the 
student-level background variables weighted by the student-level regression 
coefficients, plus error. Preliminary analysis indicated that little would be gained 
by allowing the within-classroom slopes to vary randomly or by modeling their 
variation. 

These models center observations around classroom means. Without 
group-mean centering, the predictor variance within and between classrooms 
would be confounded. Centering eliminates confounding of the predictor 
variance between and within classrooms. Centering also makes the model's 
coefficients straightforward estimates of the within-classroom and between- 
classroom effects (e.g., Bryk and Raudenbush, 1992). 

We began with the assumption that all variables that survived screening by 
EDA would be included in the models. Including some that survived the EDA, 
however, resulted in numerous small and statistically non-significant parameter 
estimates. We therefore constructed models based on what could be called a 
‘judgmental stepwise' procedure, in which we began with a null model (i.e., a 
model including nothing but an intercept), built up to a more complex model, 
and then pared back to a more parsimonious model based on the size and 
significance of coefficients.(Note 3) In general, we opted to include variables 
that were only marginally significant or that failed to reach significance by a 
modest amount, leaving it to the reader to discount them, provided that their 
inclusion did not markedly change the coefficients of other variables. In 
addition, because our classroom-level variables are aggregates of student-level 
variables, we included at both levels any variable that was significant at either 
level. 

The statistics normally reported from hierarchical models — intercepts and 
regression coefficients at each level of aggregation — are sufficient for predicting 
means but not for comparing variance of performance across countries. For 
example, at the classroom level, the estimated effect of the proportion of students 
living with their fathers indicates how much, on average, the classroom mean 
score would increase if the proportion increased from 0 to 1, but it does not 
indicate how much of the variability among classroom mean scores is 
attributable to this factor. Therefore, we also present a summary of the variance 
accounted for by the predictors at each level, expressed as the absolute value of 
the predicted variance, the percentage of variance predicted within level, and the 
percentage of total variance predicted. 
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Decomposing Performance Variation 

We first give a detailed discussion of the models for the US. This 
discussion serves as a template for evaluating the results of the other models. 

We then compare the results from the four countries. 

The final two-level model of mathematics scores in the US contained only 
five variables at each level: the number of books in the home, the presence of a 
computer in the home, the presence of the father in the home, the academic press 
variable, and student age. The square of age was included because of 
nonlinearities in the relationships between age and scores that became apparent 
in the exploratory data analysis. Each of these variables was at least marginally 
significant at one of the two levels. 

The importance of these predictors can be evaluated several ways. One 
can look at the significance and impact of the individual coefficients within each 
level, the relative significance or impact of the coefficients across levels, and the 
total predictive power of the coefficients at each level. These three views are 
each described in turn. 

Within classrooms in the US, the strongest effects were those of the 
number of books, the academic press variable, and students' age (Table 4). The 
effects of having a computer and the father living at home were both smaller and 
non-significant. Comparisons of these parameter estimates, however, is clouded 
by their imprecision. Confidence bands around most of these estimates were 
wide (see Appendix B). 



Table 4 

Two-Level Models of Mathematics Scores 



Variable 


United States 


France 


Hong Kong 


Korea 


Intercept 


-351.7 


592.6 


-424.8 


27.9 


Within class (b ) 










Number books 






0.3 


20.2** 


Computer present 


4.4 




-3.8 


10.9** 


Father present 


1.7 


8.9* 


-7.4 




Mother's education 




4.6 






Father's education 








9 5** 


Press 


9.6** 


8.6* 


10.3** 


36.2** 


Age 


- 14 . 4 ** 


-18.2** 




-6.0 


Age^ 


-6.9 


-0.6 




-14.8** 


Bom in Country . 






_19 1** 




Between-class ( c ) 










M Books 


45.5** 




44 1** 


16.2* 


M Computer 


37.2* 




89.8* 


44.5** 


M Father present 


90.3** 


59.5** 


326.9** 




M Mother's education 




26.4** 






M Father's Education 








18.8** 


M Press 


43.2** 


45.0** 


174.5** 


47 4 ** 
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MAge 33.9 -23.0* 20.5 



M Age^ 


-149.4 


-23.3 




-26.2* 


M Bom in Country 






-44.7 




Residual variances 










r (within) 


4570.4 


4040.8 


5485.0 


9290.6 


t (between) 


766.2 


554.7 


1406.2 


48.0 


NO l b. All estimates ot signiticance 


retlectjac 


kJcnited estimates: 



* p<.05 ** P<.01 

The effects of these estimates can be compared to the distribution of scores 
to provide a concrete estimate of their size. For example, in the US, the 
estimated student-level effect of the number of books was 7.9. This variable had 
five categories. The model predicts that holding constant the other variables, the 
mean difference between students in the lowest and highest categories would be 
32 points, roughly one-third of the standard deviation of mathematics scores, 
which was 89 points in this subsample. The press coefficient was larger, but 
most students were concentrated within two categories of either of the press 
variables, and the effect of being in the higher of these two categories, relative to 
the lower of them, was only about one-tenth of a standard deviation. The age 
coefficient was significant and negative, suggesting that either retention or late 
entry of slower learners have a larger impact than maturational effects. 

At first glance, the estimated effects at the between-classroom level 
(preceded by an "M," for "mean," in all tables) appear much larger than the 
coefficients at the within-classroom level. However, the standard errors of the 
estimated between-class coefficients are generally large, and the t statistics of the 
between-class coefficients are on average only modestly larger than those of the 
within-class estimates. 

Nonetheless, in the US, there are some striking differences between the 
within- and between-class estimates. The presence of the father in the home had 
a non-significant and near-zero relationship to scores within classrooms, but the 
percentage of fathers in the home showed a substantial relationship to classroom 
mean scores. On average, the estimated within-classroom effect of having the 
father present was less than 2 points, roughly 2 percent of a standard deviation. 
Classrooms in our grade 8 mathematics model sample ranged from 15 to 100 
percent of fathers present. Holding other variables constant, going from one 
standard deviation below the mean to one standard deviation above on the scale 
of proportion of fathers present (from .50 to .82) would predict an increase in 
mean scores of about one-third of a standard deviation. 

The difference in predictive power at the within- and between-classroom 
levels in the US becomes clearer if one compares the variance accounted for by 
variables at each level. In this model, 59 percent of the total variance in scores in 
the US was within classrooms, while the remaining 41 percent was between 
classrooms (Table 5). The five variables in the model predicted about 77 percent 
of the between-classroom variance but only 4 percent of the within-classroom 
variance. The predicted between-classroom variance was 2,532, while the 
predicted within-classroom variance was only 198. Thus, the five between- 
classroom variables accounted for 3 1 percent of the total variance of 
mathematics scores [2532/(3299+4769)], while the five within-classroom 
variables accounted for only 2 percent of the total variance. 

Table 5 

Total and Predicted Variance in Mathematics Scores at Each Level 
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United States 


France 


Hong Kong 


Korea 


Share of 
Variance 


Within 


Between 


Between 


Within 


Between 


Within 


Between 


Within 


Total at 
level 


3299 


4769 


1356 


4232 


4543 


5557 


799 


10722 


Percent 
at level 


41 


59 


24 


76 


45 


55 


7 


93 


Predicted 

by 

variables 
at level 


2532 


198 


801 


191 


3137 


73 


751 


1431 


Percent 
at level 
predicted 
by 

variables 
at level 


77 


4 


59 


5 


69 


1 


94 


13 


Percent 
of total 
predicted 
by 

variables 
at level 


31 


2 


19 


3 


31 


1 


7 


12 



One surprising finding in the multilevel model for the US was the lack of 
importance of mother's and father's education, which are generally considered to 
be among the strongest predictors of student performance in the US. Parental 
education did not have large enough effects to warrant keeping either variable in 
the model. Alternative models (for example, one in which the TIMSS parental 
education categories were entered as dummies) produced the same result. To 
explore this, we conducted additional analyses of TIMSS and the base year of the 
National Education Longitudinal Study (NELS-88), modifying our TIMSS 
model in several ways to make it as comparable as possible to the model we 
analyzed in NELS. This comparison suggested that several factors contributed 
to the unimportance of maternal education in our TIMSS model, including the 
use of a single classroom per school and the inclusion of the academic press 
variable. However, much of the difference remained unexplained and appears to 
be a result of unknown characteristics of the TIMSS database. When nearly 
identical models were analyzed in TIMSS and NELS, in both cases using schools 
rather than classrooms as the level 2 unit, the level 1 and level 2 parameters for 
maternal education were both less than half the size in TIMSS as in NELS. 

None of the final models fully matched any other in terms of the variables 
included (Table 4). Only a single variable, academic press, appeared in the final 
models for all countries. The final models for Hong Kong, Korea, and the US all 
included variables for number books in the home, a computer in the home, and 
academic press. The model for Hong Kong, however, included a variable for 
father present in the home but excluded age, which was included in both the US 
and Korea. The model for Korea included age but excluded presence of a father, 
which was included in the other two countries. The model for Hong Kong 
included a variable for bom in country, and, the model for Korea included a 
variable for father's education; neither of these variables was included in the 
models for any other countries. The model for France was was the only model 
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that excluded variables for the number of books or computer present and was the 
only one to include mother's education. 

Although some of the coefficients were similar in magnitude across 
countries, others differed markedly. For example, the student-level (within- 
classroom) coefficients for press were similar in the US, France and Hong Kong: 
9.6, 8.6 and 10.3, respectively. The between-classroom coefficients for this 
variable were 43.2, 45.0 and 47.4 for the US, France and Korea. In contrast, the 
between-classroom coefficient for the press variable in Hong Kong was 174.5, 
several times as large as the coefficients for the same variable in the other 
models. However, as explained below, we do not place great confidence on 
specific parameter estimates, and this estimate in Hong Kong may be seen as 
implausible. 

Although the variables in the models and the effects of those variables 
differed across countries, the models in all countries were consistent in 
predicting most of the variance between classrooms but little of the variance 
within classrooms (Table 5). This prediction of between-classroom variance 
ranged from 59 percent in France to 94 percent in the Korea, and the prediction 
of within-classroom variance ranged from 1 percent in Hong Kong to 1 3 percent 
in Korea. The prediction of within-classroom variance in Korea, while a modest 
1 3 percent, is several times as strong as in any other country; the next strongest 
prediction was 5 percent of the within-classroom variance in France. 

The consistency of this strong prediction of between-classroom variance is 
all the more striking in the light of the sparseness of the models and the weak 
measurement of social background. Our models included few predictors. The 
variables available in TIMSS do not necessarily include those that researchers in 
participating countries would suggest are the most important predictors of 
achievement. For example, TIMSS does not include income, race/ethnicity, or 
inner-city location, all three of which are known to be important predictors of 
performance in the US. Similarly, the National Research Coordinator for Korea 
indicated that income, type of community (urban, suburban, rural) and 
geographic region are all somewhat correlated with performance in Korea (Im, 

1 998). In addition, the selection of variables for use in the models was 
constrained in some instances by problems with the data. 

Thus, the variables included in the models were a potentially weak proxy 
for those that would best show the relationships between score variance and 
background variables in each country. It is possible that the use of a stronger set 
of predictors would have substantially increased the percentage of variance 
predicted at one or both levels, particularly the within-classroom level, at which 
our prediction was very weak. We cannot determine whether this is the case, 
however. In the general case, the degree of prediction may not be substantially 
lessened by the weakness of collinear predictors if enough of them are used in 
the model (e.g., Berends and Koretz, 1996). 

We have less confidence in the specific parameter estimates we obtained, 
particularly in cases in which the estimates varied markedly among countries. 
There are several reasons for this caution. First, as noted earlier, parameter 
estimates in multi-level models are often quite sensitive to specification 
differences (Kreft and DeLeeuw, 1998), and our selections of variables were 
necessarily somewhat happenstance, constrained as they were by the limitations 
of the TIMSS database. Models that included additional variables (such as 
family income) or better-measured constructs might have yielded substantially 
different estimates of the parameters in our models. Second, EDA showed that 
some variables behaved quite differently across countries. Other 
operationalizations of these constructs might have altered these differences and 
might therefore have produced different parameter estimates. 

To test the importance of the particular selections of variables in our final 
models, we ran a constant, minimal model in each of the four countries. 
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including the individual and aggregate values of number of books, computer 
present, press, age, and age squared. This fixed model predicted almost as much 
of the variance in performance as did our final models, which were selected to 
optimize prediction in each country and subject (Table 6; compare Table 5). 

This suggests that predicted variability is somewhat invariant to the variables 
included in the model. 



Table 6 

Percent of Variance at Each Level Predicted by Fixed Model 





Mathematics 


Between Classroom 


Within Classroom 


United States 


72% 


4% 


France 


54 


4 


Hong Kong 


67 


1 


Korea 


86 


12 



Differences in the strength of prediction across the four countries therefore 
may be substantively more important than differences in parameter estimates. 
One striking difference in prediction becomes apparent when one looks at the 
prediction of total variance rather than within-level variance. In the US and 
Hong Kong, roughly one third of the total variance is predicted by the models, in 
both cases largely because of variation in between-classroom predictors (Table 
7). The models predict much less of the variance in France (18 percent) and 
Korea (19 percent). 



Table 7 

Percent of Total Variance Predicted by Predictors 
at Each Level, Final Models 





Between Classroom 


Within Classroom 


Both 

Levels 


United States 


31% 


2% 


34% 


France 


14 


3 


18 


Hong Kong 


31 


1 


32 


Korea 


7 


12 


19 



NO I E: tntnes may not sum to totals because ot rounding. 



The four countries also differ in terms of the relative predictive power of 
the models between the student and classroom levels. Again, the US and Hong 
Kong are very similar: almost all of the predicted variance in each country is 
attributable to between-classroom variation in the predictors (Table 7). France 
and Korea, however, differ in this respect, even though the percentage of total 
variance predicted at both levels is nearly identical in the two countries. In 
France, most of the predicted variance is attributable to the classroom-level 
predictors, and France differs from the US and Hong Kong in that the prediction 
is much weaker at the classroom level. In Korea, in contrast to all three other 
countries, more of the total prediction is due to within-classroom variation in 
predictors. This can be seen as a reflection of two factors. First, even though the 
model predicted only a modest percentage of the within-classroom variance in 
Korea, the predicted percentage was considerably larger than in the other three 
countries. Second, a larger percentage of the total variance lies within 
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classrooms in Korea (93 percent) than in France (76 percent), the US (59 
percent), or Hong Kong (55 percent). The product of these two percentages, 
which is the percent of total variance predicted by within-classroom predictors, is 
therefore much larger in Korea than in the other countries. 

There are several possible non-exclusive explanations for these cross- 
national differences in predicted variance. First, the fixed model and our final 
models may be a better selection of variables for some countries them for others. 
Changing to a fixed set of variables drawing fi'om the variables in our set did not 
have much of an impact, but it is possible that including other variables would 
have. Second, taking our models as a given, stronger prediction in one country 
than in another could stem from larger estimated effects of some variables in the 
model, greater variability in the predictors themselves, or both. 

Stronger prediction of scores could reflect stronger partial relationships, 
greater variance in the predictors themselves, or both. To explore this, we 
partitioned the variance in the predictors themselves into within- and between 
classroom components. We then compared the amount of variance in the 
predictors to the amount of predicted variance in scores. 

The greater prediction of score variance within classrooms in Korea 
compared to the US appears not to stem from differences in the variability of 
predictors. Within classrooms, all of the predictors other than age (which 
matters less because it is a weak predictor of scores) showed roughly similar 
variance in the US and Korea. This, in conjunction with the larger parameter 
estimates reported for Korea earlier, indicate that the stronger within-classroom 
prediction in Korea stems from stronger partial relationships within classrooms 
between background variables and scores. 

The contribution of predictor variance to the difference between France 
and the US in the prediction of between-classroom score variance, however, is 
ambiguous. France shows less between-classroom variance in two predictors, 
number of books and computer present, and the former is a relatively powerful 
predictor of score variance in France. On the other hand, France shows much 
more between-classroom variance in age, and age is also a strong predictor of 
score variance. 

Recall that although Hong Kong is similar to Japan and Korea in terms of 
its overall mean and standard deviation, it is similar to the US - and strikingly 
different from Japan and Korea - in terms of the decomposition of variance into 
within- and between-school components. Hong Kong is also very similar to the 
US in terms of the predictive power of the models both within and between 
classrooms. Hong Kong and the US are also similar in terms of the within- and 
between-classroom variance of the predictors themselves, with the exception of 
age. 

Conclusions 

This study was prompted in part by a widespread view that performance 
variance in the US is unusual. This view has sometimes been made explicit - for 
example, in Berliner and Biddle's assertion that "The achievement of American 
schools is a lot more variable than is student achievement from 
elsewhere" (1995, p. 58). In other instances, this view of variability is implicit, 
as when the scores for US states or districts are compared to national averages 
from other countries. In response, we asked whether the distribution of 
performance in the US is anomalous, how the variance in performance is 
distributed in the US and other countries, and how well background factors can 
predict that variation. 

TIMSS suggests strongly that the variation in performance in the US is not 
anomalous. In Population 2, the US variance is large but not exceptional in 
science and more nearly average in mathematics. Contrary to some expectations. 
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the distribution of scores is not particularly skewed in the US, and in eighth- 
grade mathematics, it is right- rather than left-skewed. Moreover, differences 
among countries in the variance of performance do not clearly follow stereotypes 
about their homogeneity. Socially homogeneous Japan, for example, shows a bit 
more variation than the US in mathematics, while socially heterogeneous France 
shows considerably less. 

When performance variance is broken into within- and between-classroom 
components, however, the story becomes more complex. The US, Australia, 
Germany and Hong Kong show one pattern, in which nearly half of the variance 
lies between classrooms. Japan and Korea lie at the other extreme; most of their 
variance lies within classrooms, while very little lies between. The result is that 
classrooms in Japan and Korea resemble each other in terms of mean 
performance much more than do classrooms in the US, Germany, Hong Kong, 
and Australia. France falls between these two poles. By the same token, 
students in the typical classrooms in Japan and Korea show much greater 
variability in performance than do their counterparts in the US, Germany, Hong 
Kong, and Australia. 

While the US is similar to many other countries in the overall variability 
of student performance in mathematics and is similar to several others we 
investigated in the decomposition of performance variation within and between 
classrooms, TIMSS does not fully address the reasonableness of Berliner and 
Biddle's (1995) assertion that US schools are far more variable than are schools 
elsewhere. Of the countries we considered, only the US and Australia provided 
samples that allow one to separate between-classroom and between-school 
variance. For example, if tracking is entirely absent in Japan and Korea, 
classrooms within schools should be randomly equivalent. In this case, much of 
the between-classroom variance in these countries might lie between schools - in 
comparison to the US and Australia, where our preliminary analysis found that 
most of the between-classroom variance lies within schools. However, only a 
sample that includes multiple classrooms per school would permit testing this 
hypothesis. 

What do the present findings imply about the reasonableness of comparing 
means for US states and districts to averages for other nations? We cannot fully 
mswer that question because the TIMSS design does not yield evidence 
pertaining to districts or states in the US or about similar units in other countries, 
such as German Lander. However, the wide dispersion of classroom means in 
Australia and Germany, and the smaller but still substantial dispersion of means 
in France, suggests that these comparisons may be misleading. Just as some 
states in the US compare more favorably than do others to means of other 
countries, some areas in those other countries are likely to score markedly better 
than the averages for those countries. In contrast, classrooms in Japan and Korea 
vary much less in average performance, so comparisons between US states and 
the means in Japan and Korea may be more meaningful. However, even in 
Korea and Japan, the standard deviations of classroom means are substantial, and 
the standard deviation of school means, which cannot be estimated from TIMSS, 
may be sizable as well. 

Our analyses cannot identify causes of the cross-national differences we 
found, but they raise a number of intriguing possibilities that warrant further 
investigation. One question is what factors might underlie the patterns in Korea: 
little total variance between classrooms and an unusually large amount of 
predicted variance within classrooms. 

One possible contributor to the differences between the US and Korea is 
stratification of students in terms of ability. This hypothesis is consistent with 
the differences between the US and Korea in terms of both the decomposition of 
variance and the ability of the models to predict the within-classroom variance. 
We know that Korea's policy is not to track students into classes by ability in 
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eighth-grade mathematics (Im, 1 998). If schools as well as classrooms are 
relatively little stratified in Korea in terms of background factors associated with 
student performance, then more of the relevant variance of these background 
variables may lie within classrooms in Korea than in France, the US, or Hong 
Kong. Note that the total variance in the background factors included in the 
fixed model is not larger within classrooms in Korea than in the US. However, 
more of the variance that predicts student performance may lie within classrooms 
in Korea. In contrast, in countries like the US, the combination of residential 
stratification and tracking would result in much of the relevant variance of these 
background variables lying between classrooms rather than within them. 

However, other factors, such as instructional differences, might also 
contribute to the differences between Korea and the other countries examined. 

For example, instruction might vary less among classrooms in Korea than in 
Hong Kong or the US. This might help explain the lack of performance 
variation between classrooms. Instructional factors might also contribute to the 
greater within-classroom predictive power of background factors in Korea. 

Although many current US reform efforts aim for both higher standards and 
greater equity of outcomes, it is possible that all other factors being equal, a very 
high level of standards could increase score variance, as the more able students 
might be better able to take advantage of more difficult material. Curriculum 
differences might also correlate differently with background factors from one 
country to another. If curriculum differences are less highly correlated with 
background factors in Korea than in the US, that too could contribute to the 
patterns we found. 

The results for Hong Kong also raise interesting questions. Four Asian 
countries, Singapore, Korea, Japan, and Hong Kong, ranked highest in grade 8 
mathematics in TIMSS. Hong Kong is also similar to Japan and Korea, but not 
Singapore, in terms of its simple standard deviation of scores. Our results, 
however, showed that in both the decomposition and prediction of performance 
variation, Hong Kong is very similar to the US and strikingly different from 
Korea and Japan. Hong Kong is also similar to the US in terms of the 
decomposition of the variance of predictor variables. Further investigation of 
factors that might cause Hong Kong to resemble other highly developed Asian 
countries in some respects but the US in other respects could help avoid 
simplistic explanations of cross-national differences in performance. 

Finally, several aspects of performance variation in France - the relatively 
small overall standard deviation of scores, and the small total and predicted 
between-classroom variance - could have important implications for policy. As 
noted earlier, it is not clear from our results whether lesser between-classroom 
variation in predictors contributed to this, but decompositions of predictor 
variance did not suggest that this was a major factor. Some observers maintain 
that the French curriculum is highly standardized, even compared to that of many 
other countries with national curricula. If so, that uniformity could contribute to 
both a smaller between-classroom variance. In addition, by weakening any 
correlations between curricular variables and social background, uniformity of 
curriculum could also lessen the prediction of score variance by background 
factors. - 

Further analysis of TIMSS data may help shed light on these questions. 

For example, the present analysis could be expanded to incorporate instructional 
and curriculum variables as well as background factors. The TIMSS data, 
however, will not be sufficient to address certain key aspects of these questions. 

They cannot provide useful data about variations in larger aggregates, including 
schools and states (and their equivalents). Moreover, in most countries, TIMSS 
collected very little information about stratification, either within or between 
schools. These gaps could be addressed either by modifications of future 
international surveys or by the use of smaller, more focused studies in selected 
countries. 
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Notes 

1 . A number of studies have shown that even older students often provide 
reports of background variables that are inconsistent with those of their 
parents. For example, Kaufman and Rasinski (1991) showed that only 
roughly 60 percent of eighth-grade students in the National Education 
Longitudinal Study (NELS-88) agreed with their parents about their 
parents' educational attainment (Kaufman and Rasinski, 1991, Table 3.2). 
A study of Asian and Hispanic students in NAEP found similar results for 
middle-school students but found that fewer than half of third-grade 
students agreed with their parents on this variable (Baratz-Snowden, 
Pollack, and Rock, 1988). 

2. Note that the shape of the distributions depend on the mix of items 
included in the assessment. For example, it is possible that including a 
larger number of easy items in the assessment would have stretched the 
left-hand tails of these distributions, particularly the lower tail of the US 
distribution. 

3. This is in contrast to traditional stepwise or other empirical subsets 
procedures, in which criteria specified a priori, such as F-for-inclusion, are 
applied algorithmically. 
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Appendix A 

Description of Variables 



This Appendix describes the source of the principal variables used the models 
presented in this report. 



Name 


TIMSS name 


Notes 


Math score 


BIMATSCR 




Father 

present 


BSBGADU2 


Age 


BSDAGE 


Books in 
home 


BSBGBOOK 


Sometimes entered as a single variable, if test of 
linearity warranted. 


Computer in 
home 


BSBGPS02 




Press 


composite 


Mean of BSBMSIP2 and BSBMMIP2 when both 
were present; either variable if only one present 


Mother's 

education 


BSBGEDUM 


Sometimes recoded as noted in text; sometimes 
entered as a single variable, if test of linearity 
warranted 


Father's 

education 


BSBGEDUF 


Sometimes recoded as noted in text; sometimes 
entered as a single variable, if test of lineenity 
warranted 


Born in 
country 


BSBGBRNl 
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Appendix B 

Confldence Limits for Parameter Estimates 
Two-level Models 



Parameter estimates are the same as those reported in the body of the article. 
Jackknifed estimates of lower and upper 95 percent confidence limits are in 
parentheses under each parameter estimate. 



Variable 


United States 


France 


Hong Kong 


Korea 


Intercept 


-351.7 


592.6 


-424.8 


27.9 




(-884.9, 

181.5) 


(289.8, 

895.4) 


(-708.6, - 
141.0) 


(-660.5, 

716.3) 


Within class ( b ) 










Number books 


7 9** 




0.3 


20.2** 




(5.6, 10.3) 




(-2.0, 2.7) 


(16.7, 23.7) 


Computer present 


4.4 




-3.8 


10.9** 




(-2.7,11.4) 




(-10.0, 2.5) 


(3.2, 18.7) 


Father present 


1.7 


8.9* 


-7.4 






(-4.9, 8.3) 


(0.9, 17.0) 


(-19.7, 5.0) 




Mother's education 




4.6 










(1.2, 8.0) 






Father's education 








9.5** 










(5.5, 13.5) 


Press 


9 6** 


8.6* 


10.3** 


36.2** 




(4.4, 14.7) 


(0.5, 16.7) 


(4.6, 16.0) 


(27.9, 44.5) 


Age 


-14.4** 


-18.2** 




-6.0 




(-21.1, -7.7) 


(-24.6, 11.7) 




(-18.4, 6.4) 


Age^ 


-6.9 


-0.6 




-14.8** 




(-14.2, 0.5) 


(-6.0, 4.7) 




(-26.0, -3.7) 


Bom in Country 






-19 1** 










(-30.1, -8.2) 




Between-class ( c ) 










M Books 


45.5** 




44 1 ** 


16.2* 




(30.7, 60.2) 




(11.7, 76.6) 


(1.7, 30.7) 


M Computer 


37.2* 




89.8* 


44.5** 




(3.6, 70.9) 




(5.4, 174.1) 


(12.5, 76.4) 


M Father present 


90.3** 


59.5** 


326.9** 






(47.4, 133.2) 


(14.0, 104.9) 


(151.8, 502.1) 




M Mother's 
education 




26.4** 










(16.2,36.7) 






M Father's 
Education 








18.8** 
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(7.7, 30.0) 


M Press 


43.2** 


45.0** 


174.5** 


47 4** 




(9.0, 77.4) 


(14.3, 75.5) 


(103.5, 245.4) 


(13.1,81.7) 


M Age 


33.9 


-23.0* 




20.5 




(3.2, 64.6) 


(-43.1, -2.8) 




(-27.6, 68.5) 


MAge^ 


-149.4 


-23.3 




-26.2* 




(-223.8, - 
75.0) 


(-54.7, 8.0) 




(-50.0, -2.4) 


M Bom in Country 






-44.7 










(-119.9, 30.4) 




Residual variances 










r ^ (within) 


4570.4 


4040.8 


5485.0 


9290.6 


t (between) 


766.2 


554.7 


1406.2 


48.0 
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El patron de cooperacion como garantia en la 
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Resumen 

El presente articulo analiza como el patron de cooperacion 
sobre el nivel educative de Formacion Profesional entre 
diferentes instancias asegura una Formacion Profesional 
de calidad en la medida en que se produce un reparto de 
competencias efectivo y regulado. Para este analisis, se 
aplica la metodologia comparada de Garcia Garrido sobre 
el modelo espanol y aleman de Formacion Profesional y 
se revisan topicos educativos dentro de cada uno de los 
dos sistemas educativos de Formacion Profesional 
relacionados con la administracion, gestion y supervision 
del sistema en ambos paises tanto en el nivel central como 
en el regional. 
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Abstract 

This article analyzes how the patterns of cooperation 
among different institutions ensure quality professional 
preparation in terms of the production of an effective and 
regulated distribution of competencies. For this analysis 
we apply the comparative methodology of Garcia Garrido 
to the Spanish and German models of professional 
development; and we review educational issues related to 
the administration, management, and supervision of the 
professional preparation system in the two countries both 
at the central and regional levels. 

Introduccion 

El estudio de la Formacion Profesional (a partir de ahora, FP) resulta 
siempre complejo, en buena medida porque se sigue considerando un 
nivel educativo extremadamente heterogeneo, al concebirse como una 
especie de bisagra que une el sistema educativo y la empresa. Estas 
caracteristicas le confieren singularidad a la vez que gran autonomia 
como tramo educativo, configurandose de manera diferente en fiincion 
del pais en que esta se situe. En efecto existen fuertes 
condicionamientos en fiincion de las diferentes prioridades politicas y 
educativas de los paises desde los que se planifica, gestiona y pone en 
marcha la FP, puesto que mientras que algunos defienden la 
consolidacion de los sistemas FP reglada, otros prefieren fortalecer la 
formacion continua de los trabajadores, y otros mas los cauces post- 
obligatorios y aiin los ocupacionales. 

En intima conexion con el modo de comprender y asumir la 
orientacion que los sistemas de FP poseen en un determinado pais, el 
estudioso de la educacion se encuentra con la necesidad de analizar la 
politica educativa que rodea a la supervision, gestion y administracion 
de la FP, puesto que en estas hallara la justificacion de buena parte de 
las decisiones que en esta materia se planifican y llevan a cabo. 

A partir de un analisis asi entendido, la FP en el ambito europeo se 
articula a traves del reparto de responsabilidades entre los 
interlocutores sociales implicados en este nivel educativo, y desde el 
ambito local hasta el central: gobiemos y ministerios, patronales (en 
representacion a la empresa), camaras profesionales y sindicatos. 

De una forma mas concreta, una de las claves sobre las que se pretende 
reflexionar e incidir estrategicamente en este articulo consiste en que 
el patron de cooperacion entre las diferentes instancias en la FP 
espanola asegura una FP de calidad en la medida en que se produce un 
reparto de competencias efectivo y regulado, como ocurre en el 
modelo aleman. Dicho de otro modo, se piensa que la cooperacion es 
una de las premisas que mejor se adapta al modelo de politica 
educativa existente tanto en Espana como en Alemania, al aplicarse a 
la realidad autonomica y, al mismo tiempo, predicarse hoy en los 
modelos federales, ya que, como senala Puelles, “implica una decision 
generalizable a todas aquellas materias que, como la educacion, son 

objeto constitucionalmente de una competencia compartida”*. 

Para conseguir los objetivos marcados, se aplicara sobre los contenidos 
una metodologia comparada, a fin de poder comparar cual es el estado 
de la cuestion establecido en torno a dos paises elegidos como objeto 
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de estudio: Espana, por un lado, y Alemania, por otro, a traves de un 
metodo que se adapta a su vez a la metodologia comparativa de Garda 

Garrido^. La metodologia consiste en la descripdon explicativa de los 
datos correspondientes a las unidades de comparadon; la 
yuxtaposicion de datos (mediante unas tablas comparativas), a traves 
de la cual podran perfilarse cuales son las semejeinzas y las diferendas 
estableddas en torno a las categorias que se habran prefijado, para 
desembocar, finalmente, en unas conclusiones acerca de los resultados 
encontrados. 

En el transcurso de las proximas paginas, por tanto, se profundizara 
sobre los elementos de los que depende dicha cooperadon, a la vez 
que se incidira en la organizadon politica de ambos paises. Interesa, 
igualmente, insistir en cuales son los aspectos que separan a la politica 
educativa de Espana de la llevada a cabo sobre Alemania, para poder 
extraer una serie de conclusiones valiosas, fruto del analisis 
comparado. 

1. Revision del marco teorico. La administracion, gestion y 
supervision del sistema FP en Espana y Alemania. 

A continuacion se revisardi los contenidos de los modelos de FP 
correspondientes a cada uno de los paises objeto de estudio. Se 
comenzara con la exposicion de Espana a la que seguira el modelo 
aleman. 

1.1. Administracion, gestion y supervision del sistema de FP en 
Espana 

a) Constitucion espanola y administracion educativa. 

La Constitucion de 1978 contiene todas las directrices basicas de la 
legislacion en el area concreta de educacion. En ella se reconoce a la 
educacion como uno de los derechos fundamentales que el Estado 
debe garantizar, y tambien recoge otros derechos basicos referidos a la 
educacion y a las tareas que, en materia educativa, corresponde al 
Gobierno Central y las Comunidades Autonomas (CC.AA). 

La Constitucion establece la distribucion de responsabilidades entre el 
Estado y las Comunidades Autonomas en las que el pais esta dividido. 
En materia de ensenanza (educacion preescolar, primaria y secundaria, 
FP reglada y ensenanza superior), la Constitucion estipula una serie de 
ambitos sobre los que el Estado tiene competencias exclusivas y otros 
en los que las Comunidades Autonomas pueden asumir 
responsabilidades. Existen actualmente siete de las diecisiete 
Comunidades Autoriomas en eSta situacion, y se preve que todas ellas 
se vayan haciendo cargo de dichas competencias de forma gradual, lo 
que llevara como consecuencia el hecho de que el sistema educativo 
espanol esta gestionado por diecisiete administraciones bajo la 

coordinacion del Ministerio de Educacion y Ciencia (MEC)^. Aquellas 
Comunidades Autonomas que todavia no ban asumido competencias 
plenas estan desarrollando de forma progresiva nuevas actividades 
educativas mediante acuerdos de colaboracion con el MEC. 

La Constitucion concede a la administracion central del Estado una 
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competencia exclusiva en los siguientes ambitos de la educacion, y por 
lo tanto tambien en lo que a FP reglada se refiere. Asi, puede afirmarse 
que, administrativamente, el Estado, este tiene competencias en varies 
aspectos: entre los mas importantes, destacan la regulacion y puesta en 
practica de las condiciones basicas que atanen al principio 
fundamental del derecho a la educacion, establecimiento de la 
normativa general del sistema educative (duracion de periodos, 
niveles, ciclos, grades, niveles, materias de especializacion, requisites 

de los examenes, etc.); y la programacion de la ensenanza'^. 

La unica administracion educativa central con competencias de nivel 
nacional es el Ministerio de Educacion y Ciencia (MEC) que 
coordinara las actividades con las realizadas por las administraciones 

educativas autonomicas^. Dentro de esta institucion, dirigida por un 
Ministro que es a su vez asistido por otros organismos directives, en el 
MEC se distinguen varies suborganismos, entre los cuales destaca la 
Secretaria de Estado en Educacion, que incluye a diferentes 
Direcciones Generales: las Direcciones Generales de Renovacion 
Pedagogica, de Centres Escolares, de FP Reglada y Promocion 
Educativa. 

El Ministerio de Educacion y Ciencia es considerado por De Puelles 
Benitez como una organizacion 'hibrida', ya que “por un lado va a ser 
el organismo gestor con respecto a las Comunidades Autonomas que 
no accedieron a la plena competencia; por otro, ha asumido con 
dinamismo las flinciones autonomicas (aunque no todas al mismo 
nivel) y, finalmente ha acometido una profunda reforma del sistema 
educative, lo cual ha conllevado inevitablemente una considerable 

carga de gestion”^. 

La administracion educativa ha suffido, desde la promulgacion de la 
Constitucion hasta el momento presente, considerables 
transformaciones para ajustarse a la descentralizacion administrativa 

del Estado^. En el piano autonomico, ya se ha senalado que es 
necesario distinguir entre aquellas Comunidades Autonomas que 
hayan asumido plenas competencias en materia educativa y aquellas 

Q 

que continiian siendo gestionadas por el MEC . En este caso, las 
primeras contaran con sus propias estructuras administrativas que se 
corresponden en principio con la organizacion de la administracion 
central de la ensenanza y de la FP. Conviene destacar el hecho de que 
todas las Comunidades Autonomas cuentan con un departamento de 
Educacion (la Consejeria de Educacion), si bien estos varian en 
dimension e importancia, en funcion de la comunidad y su disposicion 
0 no de plenos poderes en materia educativa. 

De entre las competencias fundamentales de las Comunidades 
Autonomas, que inciden en la FP reglada, considerada esta como nivel 
educative, ha de destacarse la elaboracion y aprobacion de planes, 
programas de estudio y orientaciones pedagogicas que complementen 
las ensenanzas minimas fijadas por el Estado. 

Pero la dinamica de la descentralizacion no se agota en las 
Comunidades Autonomas; existe ademas un tercer nivel de 
competencia, el correspondiente a la administracion local, aunque 
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apenas tiene responsabilidades en educacion, tan solo de caracter muy 
restrictivo en la educacion preescolar y basica, y ninguna incidencia en 

la FP^. 

Con respecto al nivel concreto de FP, las responsabilidades son 
compartidas entre dos instancias: por un lado, y en primer lugar, la FP 
corresponde en un alto grado al mundo laboral, y se ejerce legalmente 
a traves de los interlocutores sociales y los consiguientes acuerdos con 
el Estado, aspecto este que garantizara la legalidad, los niveles en que 
se imparte, las condiciones minimas y la supervision de las 
competencias a las administraciones perifericas. En segundo lugar, 
recae sobre el Ministerio de Educacion y Ciencia y es en su desarrollo 
formal cuando se fomenta la participacion y colaboracion con las 
empresas (aunque quede resguardada por una solida financiacion 

publica^®). De este modo, se esta demostrando que, en materia 
educativa en general, y en FP en particular, el principio basico que ha 
de presidir la articulacion de un sistema concebido desde las 
autonomlas es precisamente el correspondiente a la cooperacion. 

De todo lo anterior, puede deducirse que nos encontramos ante una 
descentralizacion administrativa practicamente total, y una 
considerable descentralizacion politica. Esta caracteristica es la que 
permite comparamos al modelo de administracion aleman. Sin 
embargo, el sistema de administracion educativo espanol se halla mas 
propiamente en el punto medio entre un sistema federal y uno regional. 
En efecto, las competencias que la Constitucion, los Estatutos de 
Autonomia y las leyes organicas de educacion que asignan al Estado 
son importantes desde el punto de vista del poder de decision 
(competencias, que, como ya se ha senalado, recaen sobre la 
ordenacion general del sistema educativo, las ensenanzas minimas que 
integran el curriculo, etc.) Estas competencias son competencias 
substanciales, que alejan al sistema espanol del modelo federal donde 
el poder federal, la Federacion, apenas tiene competencias en materia 
educativa. 

b) Participacion social en FP espanola. 

El proceso de descentralizacion de responsabilidades no constituye una 
mera redistribucion de poderes entre los diferentes niveles 
administrativos, sino que tambien supone y a su vez se alimenta de una 
fiierte participacion social. Los organismos consultivos de nivel 
nacional responsables de garantizar dicha participacion social son el 
Consejo Escolar del Estado, y el Consejo General de la Formacion 
Profesional. 

Con relacion al primero, el Consejo Escolar de Estado, este organo 
participative se compone de ochenta miembros, incluyendo las 
organizaciones patronales y las centrales sindicales. Tiene 
competencias consultivas en relacion con la legislacion, su 
reglamentacion, la programacion general de ensenanza y la emision de 
los informes pertinentes. Existe, ademas, un Consejo Escolar en cada 
Comunidad Autonoma, en el ambito provincial y en el ambito 
comarcal, con representacion de los agentes sociales. 

Con respecto al ultimo de estos, el Consejo General de la Formacion 
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Profesional' *, este cumple una flincion primordial, tal y como es el 
asesoramiento al Gobiemo sobre cuestiones relativas tanto a la FP 

reglada como a la formacion ocupacional , asi como a la orientacion 
profesional. Tambien es el encargado de disenar y realizar un 
seguimiento de la aplicacion del Programa Nacional de Formacion 
Profesional. Sus miembros ban de representar a partes iguales a los 
agentes sociales y a los Ministerios de Trabajo y Seguridad Social. 

A los dos organismos anteriores hay que unir un tercero, la 

Conferencia de Educacion*'’: La 'Conferencia de Consejeros Titulares 
de Educacion de los Consejos de Gobierno de las Comunidades 
Autonomas y el Ministerio de Educacion y Ciencia' es el largo tltulo 
que queda resumido con el termino anterior. Esta Conferencia tiene su 
origen en la LODE, surgiendo en 1985 con tres funciones muy 
especificas: la programacion general de la ensenanza (siendo una 
competencia compartida), la coordinacion de la poh'tica educativa y el 
intercambio de informacion. 

Como ocurre con el modelo federal aleman, los Acuerdos de la 
Conferencia Permanente de Educacion, aun gozando de unanimidad, 
carecen de fuerza jun'dica de obligar y requieren de las autoridades 
estatales y autonomicas en el ambito respective de sus propias 
competencias, el uso de poderes normativos y de ejecucion para poner 
en practica las decisiones tomadas previamente por la misma. 

La Conferencia actua como maxima instancia de decision, indicando 
con ello su caracter de autentico organo de cooperacion en la linea de 
los modemos modelos federales de asociacion. Sin embargo, el hecho 
de que diez anos despues de su nacimiento no se haya regulado este 
organismo indica que posiblemente necesite un mayor impulse de cara 
a abordar nuevas fases de la educacion y a fin de poder conseguir una 
mayor complejidad y desarrollo. 

1.2 Administracion, gestion y supervision del sistema de FP en 
Alemania 

a) Principios fundamentales y bases legales del sistema de FP aleman. 

La Republica Federal Alemana es una republica democratica, federal y 
constitucional y un Estado responsable, estando a su vez formado por 

\6Lader (Estados)*^ que cuentan con su propia Constitucion y 
gobiemo. 

La Constitucion de la Republica Federal alemana, conocida como 
Grundgesetz o Ley Fundamental, fue adoptada en 1949 para 
conformar un nuevo sistema politico basado en la libertad y en la 
democracia, y modificada tras la caida del Muro de Berlin, siendo 
objeto de una reforma en septiembre de 1990, con motivo de la firma 
del Tratado sobre el restablecimiento de la unidad alemana. En su 
Preambulo puede leerse que los alemanes estan llamados a “conseguir 

en libre auto-determinacion la unidad y libertad de Alemania”’^. La 
Ley Fundamental establece, asi, que la Republica Federal alemana es 
un Estado federal democratico y social, de modo que el ejercicio de los 
poderes gubernamentales se divide entre la Federacion y los Ldder. A 
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un nivel federal, las funciones ejecutivas son responsabilidad del 
Gobiemo Federal. En cada LMer, la autoridad soberana corresponde a 
cada uno de los gobiernos y parlamentos del Land. ■, 

b) Federalismo en la administracion, gestion y supervision de la FP 

En la Repiiblica Federal de Alemania la responsabilidad con respecto 
al terreno educativo esta determinada y condicionada por la estructura 
federal del Estado. Asi, tal y como senala el Institute de Investigacion 
Max Planck, una de las peculiaridades dominantes del sistema 
educativo aleman es su muy temprana y marcada estatalidad, situando 
a este pais equilibrada y proporcionalmente entre la tradicion 
anglosajona y la francesa: la primera con un enorme influjo por parte 
de las iniciativas locales, y la ultima con una uniformidad y 

consistencia calificada incluso de “monolitica”'^. Este rasgo, 
fundamental, es el que permite definir su 'federalismo' u organizacion 
federal, definiendose desde el equilibrio en la administracion y gestion 
de la educacion alemana. 

El gobierno federal esta conformado por un total de 16 Ministerios 
federales, coordinados por el Canciller federal, figura politica que goza 
de una posicion autonoma con respecto a los anteriores. La flincion de 
este ultimo queda resumida en “su poder para determinar las lineas 
directrices de la politica general”, tal y como recoge el articulo 65 de 

la Ley Fundamental , siendo por tanto el responsable directo ante las 
mismas. Con respecto al gobierno federal y a los ministerios federales, 
este primero posee competencias educativas desde el Ministerio 
Federal de Educacion y Ciencia, Investigacion y Tecnologia al 
constituirse en responsable de la politica, coordinacion y legislacion 

con respecto a la FP establecida fuera del ambito escolar*^, asi como 
de la formacion continua, junto a otras competencias educativas, en 
otros ambitos: la asistencia profesional, los principios generales que 
rigen al sistema de educacion superior, asi como la expansion y 

construccion de instituciones de este ultimo tramo educativo*^. A este 
respecto, el topico de la educacion no es exclusive compromise del 
Ministerio de Educacidn y Ciencia, sino que ha de ser necesariamente 
compartido con otros ministerios, al comprender algunas tematicas 
cuya competencia es incumbencia y por tanto responsabilidad directa 

de los mismos . 

De forma especifica, las responsabilidades del gobiemo federal sobre 
el campo de la educacion tambien quedan definidas a partir de la Ley 
Fundamental, segiin la cual la Federacidn acarreara particularmente la 
responsabilidad sobre las regulaciones en diferentes campos de 
educacion, ciencia e investigacion. De entre las mas significativas para 

nuestro actual objeto de estudio, destacan^^: la FP en la empresa y la 
FP continua, la responsabilidad sobre el marco que rige los principios 
generales de la Educacion Superior, las regulaciones para entrar a 
profesiones legales, asi como las medidas de promocion de empleo, 
ademas de la investigacion ocupacional y del mercado de trabajo. 

La Federacion, respecto a la FP, se responsabilizara de la 
reglamentacion profesional basica y superior que se realiza en las 
empresas, dentro del marco de la legislacion economica laboral, a 
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partir de la cual los Liider defmiran y desarrollaran las disposiciones 

pertinent es en fiincion de sus propia necesidades . Dicho de otro 
modo, la FP en empresas y la FP de caracter escolar se encuentran bajo 
diferentes jurisdicciones. El Gobiemo Federal es el responsable de 
determinar la normativa relativa a la FP en las empresas, pero son los 

Liider los que establecen a su vez los diferentes planes de FP . 

Dentro del primer nivel, el Gobiemo federal delega en el Ministerio 
Federal de Educacion, Ciencia, Investigacion y, Tecnologia (BMBF) 
las areas de responsabilidad de la Federacion. Este se halla organizado 
en siete Directorados Generales, uno de los cuales, el Directorado 
General 2, bajo el epigrafe de 'Educacion general y FP', se 
responsabiliza de nuestro actual objeto de estudio. El Ministerio 
Federal de Educacion, Ciencia, Investigacion y Tecnologia ejercera las 
fiinciones de legislacion, de control y de coordinacion global, segun lo 
estipulado en la Ley de Formacion Profesional de 1969 
(Beru/bildungesetz). Otras fiinciones son las relacionadas con: el 
establecimiento de la competencia en lo concemiente a las cuestiones 
fiindamentales de la politica de formacion, a la elaboracidn de los 
Reglamentos de Formacion (Ausbildungsordnungen) para el 
aprendizaje en la empresa, a la adopcion de reglamentaciones relativas 
a la cualificacion pedagogica de los formadores, al Institute Federal de 
Formacion Profesional y a la elaboracion del Informe anual sobre FP 
{Berufsbildungsberichi), entre otras. 

A su vez, cada uno de los Ministros de Educacion y Asuntos 
Culturales de los Liider adopta provisiones legates y regulaciones 
administrativas, en los campos que son de su competencia, cooperando 
con las autoridades en cada Ldder y supervisando ademas el trabajo de 
las autoridades, de los fondos y de las instituciones bajo su tutela y a 
traves de cuerpos subordinados. Para llevar a cabo estas fiinciones se 
ban creado a tal efecto sus propios institutes de investigacion sobre la 
educacion en general, y sobre la ensenanza superior y continua, en 
particular. Los Ministerios de Educacion y Asuntos Culturales de los 
Ldderson, por tante, las principales autoridades en lo que se refiere a la 
supervision y administracion de los centres que imparten la ensenanza 
general y la FP. 

La autoridad de los Lader no solo abarca la organizacion de los centres 
escolares, el contenido de los estudios y los objetivos de la ensenanza, 
sino tambien la supervision del rendimiento de los profesores y de 
todo el resto del personal docente. Los objetivos educativos 
establecidos por la legislacion educativa se materializan en los planes 
de estudio para las diferentes materias impartidas en los niveles de 
ensenanza, que son responsabilidad de los Ministerios de Educacion y 
de Asuntos Culturales de los Liider. 

A este nivel es donde se ejerce, en la practica, el control real de la 
aplicacion de los Reglamentos de Formacion (que ban side elaborados 
de manera concertada en el seno de comisiones pluripartitas), la 
habilitacion y el control de las empresas, la organizacion de la 
necesaria cooperacion entre los representantes del mundo profesional 
(Camaras, empresarios y sindicatos) y los representantes del sistema 
educative (en el nivel de cada Liider); y la organizacidn de la 
altemancia para los diferentes sectores de la economica regional. 
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c) Colaboracion entre Federacion y Ltider. Papel estrategico de los 
interlocutores sociales 

Ademas de establecer la division de responsabilidades descrita 
anteriormente, la Constitucion Federal tambien estipula mecanismos 
de cooperacion entre la Federacion y los Leider, como actividad 
conjunta desarrollada en la construccion y expansion de los centres de 
ensenanza superior. 

En Alemania el dialogo social constituye uno de los fiindamentos del 
sistema sociopolitico, aunque las relaciones convencionales solo estan 
institucionalizadas en flincion de los diferentes sectores profesionales 

existentes^^. En general, la cooperacion de los interlocutores sociales 
variara mucho, en fiincion de los distintos sistemas educativos que 
sean objeto de estudio y analisis. En el caso de Alemania, en el que la 
FP esta integrada en el sistema de trabajo, “los interlocutores sociales 

tienen mas posibilidades de representar sus intereses’ . De este 
modo, la funcionalidad de este sistema dependera en buena medida de 
que los grupos sociales representados defiendan 'responsablemente' 
sus miras y objetivos. La cooperacion y acuerdo entre todos los que se 
hallan envueltos en la planificacion de las politicas de FP se basan en 
una estructura de instituciones, relativamente diferenciada, aunque lo 
suficientemente estable. 

La distribucion de competencias y de las estructuras de toma de 
decisiones del sistema de FP alemana resulta ser, asi, altamente 
compleja. Elio es debido a que el Estado (el gobiemo federal, los 
Liidery los cuerpos regionales publicos, en torno a comites), las 
Camaras, las asociaciones de empresarios y los Sindicatos son 
conjuntamente responsables de que la FP tenga lugeir en unos 
parametros de calidad, encontrandose con las demandas cuantitativas y 
cualitativas que la sociedad y la economia le estan exigiendo. De este 
modo, la estructura de cooperacion entre las agendas publicas que 
trabajan en la politica de FP en el nivel central, regional y local acaban 
igualmente caracterizandose por su federalismo, como resultado de 
una mixtura de las competencias regulatorias centralizadas y 
descentralizadas. Esta organizacion federalista es la que conduce a la 
diferenciacion, como ya se ha visto, entre las competencias del 
gobiemo federal, y las correspondientes a los Lader. 

El Institute Federal de Formacion Profesional {Bundesinsittut jn 
Berufsbildung -BIBB), establecido a partir de las directrices de la Ley 
de Promocion de la Formacion Profesional 

(BerufsbildungsJbderungsgestezBeiBiFG) de 1981, es en la actualidad 
el instmmento mas eficaz de cooperacion entre los empresarios, los 
Sindicatos, la Federacion y el Lader, en el piano nacional. La Ley de 
Promocion de Formacion Profesional define cuales seran sus fimeiones 

y responsabilidades. Entre ellas, se destacan : Aconsejar al Gobiemo 
Federal en cuestiones relacionadas con FP; llevar a cabo 
investigaciones bajo programas pre-definidos; mantener y publicar las 
listas de las ocupaciones reconocidas que requieren FP formal 
(Anerkanne ausbildungsberufe)', monitorizar y conceder licencias a los 
cursos de FP a distancia, asi como animar al desarrollo de esta 
modalidad de aprendizaje a traves de la investigacion constante y la 
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provision de consejos y recomendaciones^^; asistir a la preparacion de 
las regulaciones para la formacion, bajo las instrucciones del gobierno 
Federal (Ausbildungsordnungeb); compilar un informe sobre el estado 
de la cuestion en el momento presente con respecto a la FP 

{Berufsbildungsbericht) \ contribuir a la innovacion con la propuesta 

de nuevos proyectos-piloto , y, bajo las provisiones generales 
administrativas del Gobierno Federal, dar soporte institucional y 
recomendaciones para la planificacion, establecimiento y desarrollo de 
la FP en las empresas. 

Debido en buena parte a su composicion (una representacion 
igualitaria del gobierno federal, de los gobiemos de los Ladery del 
abanico de organizaciones de empresarios y sindicatos, que cristaliza 
en la igualdad establecida sobre el niimero de votos), el espiritu que 
guiaal Institute Federal de Formacion Profesional tratade limar las 
diferencias que van surgiendo en el transcurso del desarrollo y avance 
de la FP que se suceden a nivel tanto sectorial como regional. De 
hecho, aunque el BIBB es oficialmente el unico responsable de la 
formacion ofrecida por la empresa dentro del sistema dual, permite 
tambien la 'intromision' del gobierno federal, de los Ltider y de los 
interlocutores sociales, a fin de poder intercambiar puntos de vista 

on 

acerca de las cuestiones basicas que rigen la politica de FP . 

Por su parte, los Liider materializaran el control ejercido sobre la 
educacion y la cultura con la creacion de estructuras poHticas de 
cooperacion y coordinacion en materia educativa, tal y como son las 
Conferencias Permanentes de Ministros de Educacion y Asuntos 
Culturales (Stddige Konferenz der Kultusminister der Lnder in der 
Bundesrepublik Deutschland, KMK), de caracter suprarregional, con 
la fmalidad de lograr posturas conjuntas y de solucionar los asuntos 

educativos y culturales de interes comiin^ ' . La primera se establecio 
en 1948, y desde aquel momento, la Conferencia de Ministros se ha 
convertido en un foro continue de debate y toma de postura en areas 
de educacion, de formacion, de educacion superior e investigacion, 
ademas de aspectos directamente relacionados con la cultura. Las 
Conferencias de Ministros han tenido que acatar momentos y 
decisiones muy conflictivas, incluso en los anos que marcaron a la 
post-guerra. Con respecto a la FP, la Conferencia resulta tambien, y en 
este caso de un modo especial, un instrumento para la cooperacion con 

^9 

la Federacion, basada en el partnenariado . 

Tampoco ha de olvidarse el papel que cumplen los diferentes comites 
regionales que se establecen en cada Lader, cuya mision se resume en 
recomendar a los gobiemos de los Lader respecto a los intereses 
concentrados sobre la FP. De este modo, los citados comites habran de 
“unir la cooperacion entre la formacion basada en la escuela y la 
formacion provista de acuerdo con la Ley, y tratar de asegurar que 
dicha integracion es necesaria por la reorganizacion y la expansion del 

sistema educativo”^^. 
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2. Yuxtaposicion de datos. Tablas comparativas 

Tras haber repasado el estado de la cuestion de Espana y Alemania en 
materia de administracion, gestion y supervision de la FP se abordara, 
a continuacion, y tal y como se indica segiin el metodo comparado que 
se ha decidido aplicar para el presente estudio, la yuxtaposicion de 
datos. A su vez, para facilitar su estudio, el analisis del topico anterior 
se estructura en tomo a dos grandes nucleos: los principios basicos que 
se derivan de las respectivas Constituciones de ambos paises, y la 
revision realizada en tomo a los organos gubemamentales para la 
administracion, gestion y supervision del sistema de FP. Este ultimo 
vuelve a dividirse en funcion de las fiinciones y tareas que son 
competencia de tres instancias educativas: los organos centrales, los 
organos regionales y las instituciones de cooperacion central y 
regional. 



Tabla 1 

Administracion, gestion y supervision del sistema 
de FF en Espana y Alemania 



Fiincipios basicos (Constitucion) 

Semejanzas 

• Re fe r e nc ia al pr in c ip i 0 de de s c e ntr aliz ac i 6 n e due ati va. 

• Presencia de organismos de see ntraliz ados en los documentos basicos para la 
educacion. 

• Cooperacion entre el nivel central y los niveles mas descentralizados. 

Diferencias 

• Le yes con una duracion dife rente. 

• Diferencias en las competencias educativas exclusivas del Estado. 

Diferencias en las competencias educativas compartidas. 





• W- AllmatiW W" 


Ley basica mas reciente 


Ley mAs tradicional 


Competencias exclusivas del 

Estado; regulacidn de los titulos 
pro fes ion ales 


Competencias exclusivas del Estado: 
regulacidn en la formacidn en la 
empresa y la formacidn continua. 
One ntaci ones y regulacidn de 
profesiones legal es 


Competencias compartidas en 

materia de cultura, investigacion y 
lengua propia. 


Competencias compartidas en 

convenios de ensenanza y proyectos 
de investigacidn suprarregionales 



2.1. Principios basicos (Constitucion) 

La descentralizacion es un principio politico en ambos paises con 
Claras repercusiones educativas. En las Constituciones respectivas 
(Grundgesetz o Ley Fundamental alemana y Constitucion Espanola), 
las competencias que se establecen entre los poderes centrales y 
regionales son muy similares entre si. 

El peso de la tradicion en la Constitucion alemana con respecto a la 
espanola, mas reciente, se muestra como un aspecto cuanto menos 
llamativo, que debe ser tenido en cuenta. La primera corresponde a 
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1949, y apenas habia sufrido cambios con relacion a su antecesora, la 

Constitucion de Weimar (1914)^'*. En 1990 la Constitucion alemana 
bubo de cambiar, ante la reciente adhesion de nuevos Liider y la 
consideracion de nuevos elementos que han de ser modificados, 
actualizados y presentados en una Ley cada vez mas globalizadora que 
se aplica sobre la nueva realidad. 

Espana, por su parte, cuenta con una Constitucion mucho mas reciente, 
fruto de los nuevos aires de democratizacion que comenzaron a 
apreciarse en este pais a finales de la decada de los anos setenta. La 
democratizacion encuentra en la descentralizacion educativa uno de sus 
mayores aliados, tal y como se desprende de la lectura de los articulos 

mas vinculados con la FP^^: el articulo 2do. (que garantiza la 
autonomia de las nacionalidades y regiones) y el articulo 148 
(competencias que pueden asumir las Comunidades Autonomas). 

De la seleccion realizada de los articulos de ambas constituciones, cabe 
reconocer una division, manteniendo con ello una doble 
intencionalidad: por un lado, el articulado que hace referenda al 
caracter o naturaleza politica que sirve a su vez de basamento para la 
promulgacion de cuantos principios politicos (y en este caso, 
educativos) se establezcan en el pais; y por otro lado, los preceptos que 
se relacionan de forma mas estrecha con el campo de la FP, aunque 
algunos sean abordados mas directamente que otros. 

El espiritu de la primera intencionalidad queda recogido en el articulo 
2do. de la Constitucion espanola y el Articulo Iro. de la Ley 
Fundamental alemana: mientras el primero esta justificando la 
existencia de una nacion dentro de la cual existen regiones 
autonomicas, el segundo se remite a su composicion federal, bajo el 
trasfondo de los principios democraticos en los que se sustenta. 

El articulado que se refiere en ambos casos a la FP es reducido, aunque 
muy revelador de las semejanzas y diferencias de los respectivos 
sistemas. 

Para el caso espanol, lo anterior supone, en primer lugar, “garantizar el 
fomento de una politica que legitime la formacion y la readaptacion 

profesional”^^. Pero hay mas medidas, y estas se concentran en tomo al 
reparto de competencias entre el Estado y el ambito regional: a este 
ultimo se le delega el establecimiento de las competencias que se 
concentran en las Comunidades Autonomas (entre las que se encuentra 
'el ambito de la cultura, de la investigacion y de la ensenanza de la 

lengua')^^, en tanto que, en materia profesional, el Estado mantendra la 
competencia exclusiva en la regulacion de las condiciones que rodean a 

la obtencion, expedicion y homologacion de titulos profesionales . 

Para el caso aleman, se repite en cierto modo el esquema que se ha 
podido comprobar en el modelo espanol, puesto que vuelve a traducirse 
el reparto de competencias educativas en general (con alguna referenda 
directa a la FP). No obstante, se reconoce ademas un articulo especifico 
en el que se habia explicitamente de dicha cooperacion: este es el 
articulo 91, en el que se expone que existira una cooperacion entre la 
Federacion y los diferentes Ldder en materia de ensenanza e 
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investigacion, siempre que estos posean un caracter suprarregional^^. 

Anadido al anterior, dos articulos vuelven a incidir en lo que ya se 
ha adelantado: mientras el articulo 50 remite al lector a la participacion 

de los Lader en la legislacion y Administracion de la Federacion^®; el 
articulo 63 se centra en el papel de la Federaci6n, y entre sus flinciones 
se encuentra una, muy especiflca, relativa a la FP, relacionada esta con 
las responsabilidades que se ejercen sobre la FP en la empresa y la 
formacidn continua, asi como las regulaciones ocupacionales para 
entrar a profesiones legales vigentes, sin olvidar la orientacion 

profesional^^ 

Ahora bien, y de acuerdo a lo expuesto hasta el momento, el reparto de 
competencias educativas entre las instancias centrales y los diferentes 
niveles de descentralizacion que se han estipulado para cumplimentar 
funciones, en este caso, educativas, si bien se aplica de forma 
equilibrada sobre todos los niveles educativos en Espana (es decir, las 
funciones mas generales a un nivel mas central y las mas especificas a 
un nivel mas regional), en Alemania dicho reparto se realiza de manera 
diferente en funcion del nivel educativo de que se trate, tal y como ya 
se desprende de la Constitucion y se completa en las diferentes Leyes 
educativas que tienen como objeto de estudio e identificacion la FP y 
que ser^ analizadas en el siguiente apartado. 

Por lo tanto, una conclusion a la que se puede llegar y que comparten 
en cierto modo ambos paises es que la FP, asi, resulta ser un nivel 
educativo y formativo que se justifica desde el reparto de competencias 
entre diferentes instancias. Ahora bien, partiendo de este principio, 
comiin a ambos paises, habra que admitir la variante que se produce en 
Alemania, y es que ese reparto no es proporcional, como ocurre en 
Espana, en la que se entiende a la FP como una unidad integrada, de 
modo que las competencias se descentralizan y alcanzan mayores 
indices de concrecion al llegar a instancias regionales. En el primer 
pais, al contrario, tiene lugar una division 'salomonica', fruto de la 
bifurcacion que solo puede ser posible desde la consolidacion de las 
dos formas de ver y comprender la FP: la Federacion se 
responsabilizara de la parte relacionada con la parte empresarial, en 

tanto que los Ltider lo aplicar^ sobre la parte escolar'^^. 



Tabla2 

Organos centrales para la administracion, gestion y supervision 
de la educacion en Espana y Alemania 



Organos centrales 

Semejanzas - - ~ 

El gobierno constituye una figura central en la organizacion central de laFP. 

El Gobierno delega en el Ministerio de Educacion la responsabilidad y sus funciones 
administrativas y ejecutoras en materia profesional. 

Existen funciones comunes que se relacionan directamente con losprincipios basicos 
legislativos. 

Funciones de regulacion. 

Existe tambien cooperacion con otros Ministerios. 

Diferencias 

Tipologia de gobierno que se ha erigido. 

^ : .i.i tk-;.. . .i_ t: .i ; ' 
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'w'lgdliiziiuioii lliicilibl UCX XMimSlCIlO UC i:rUUUdU10Il. 

Diferencias en el ambito de regulacion de FP. 

Diferencias en los campos de competencias delMinisterio de Educacion. 
Numero de Ministerios implicados. 



K I 




Gobiemo central. 


Gobiemo Federal. 


Secretaria de Estado y Direccion 
General de Formacion Profesional 
Reglada y Promocidn Educativa. 


Director ado General 2: Educacion 
Ge ner al y F orma c io n Pr o f e s io nal . 


Regulacion de Titulos de Formacion 
Profesional. 


Regulacion de Titulos y Contrato de 
aprendizaje. 


Sobre el ambito de Formacion 
Profesional reglada. 


Sobre el ambito de la Formacion 
Profesional Continua y la formacion 
en la empresa. 


Intervencion conjunta con el 
Ministerio de Trabajo. 


Intervencion con el Ministerio 
Federal de Trabajo y Asuntos 
Sociales, Ministerio Federal de 
Asuntos Familiares y Ancianos, 
Mujeres y Jovenes. 



2.2. Organos centrales 

El Estado mantiene competencias exclusivas sobre la FP, pero en 
diferentes ^bitos dentro de la misma. Las competencias estatales 
estan abarcando mas modalidades de FP en Alemeinia que en Espana, al 
incidir la primera tambien sobre la FP continua y la FP de la empresa. 
Va a existir un firme proposito por parte del gobiemo federal 
(delegando este su poder en el Ministerio Federal de Educacion y 
Ciencia, Investigacion y Tecnologia [BMBF]) de legislar, controlar y 
organizar la educacion en sus aspectos generales, incidiendo, por 
ejemplo, en los Fundamentos legales que justifican la dinamica actual 
establecida en tomo a la Formacion Profesional; pero tambien se 
concentra sobre el resto de modalidades de FP; las regulaciones 
profesionales, los contratos de Aprendizaje (dentro del sistema dual, 
opcion mayoritariamente elegida en Alemania), la reglamentacion de la 
FP basica y superior realizada en las empresas, la Formacion Continua 
y las funciones que se relacionan de forma estrecha con la supervision, 
la planificacion y la fmanciacion de la ensenanza de las escuelas de FP. 

En Espana las competencias del gobiemo se muestran necesariamente 
mas limitadas en nuestro objeto de estudio (al no ampliar, al menos de 
forma expHcita, el ambito de sus competencias sobre la FP continua y 
ocupacional, ya que sera el Ministerio de Trabajo y Seguridad Social el 
que se encargara de materializar estas funciones). El Ministerio de 
Educacion atiende los aspectos relacionados con la normativa general 
del sistema educativo, la programacion de la ensenanza y la 
fmanciacion de la educacidn, en general, y con la regulacion de los 
tifulos profesionales, en concreto, con todo que ello comporta: 
constante revision y actualizacion de los titulos profesionales, 
evaluaciones y emision de resultados. Este ha sido, precisamente, uno 
de los cometidos que se materializa de forma mas directa en el Primer 
Programa Nacional de Formacion Profesional y se incide en el 

Segundo Programa'^^. 

Indirectamente, esta funcion puede relacionarse con una atencion que 
se halla presente de algun modo sobre la FP continua y ocupacional, si 
semantiene viva la preocupacion por la acreditacion de la experiencia. 
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tal y como se desprende tambien del ultimo programa Nacional de 
Formacion Profesional. Sin embargo, tambien hay que tener en cuenta 
que en Espana ambas modalidades se encuentran en diferentes 
Ministerios, al corresponderse la formacion ocupacional y Continua 
con el Ministerio de Trabajo y Seguridad Social. 

Y es que, ademas, existen en ambos paises competencias compartidas 
con el Ministerio de Trabajo. En Alemania se suma ademas el 
Ministerio Federal de Asuntos Familiares, Mujeres, Jovenes y 
Ancianos. De esta manera se pretende llevar a cabo una politica de FP 
energica que se conecte con determinados colectivos sociales mas 
susceptibles de sufrir problematicas de todo tipo relacionados en mayor 
0 menor parte con carencias o debilidades en la formacion recibida. 

Concluyendo con las funciones del gobiemo, el rasgo que mas 
divergencia presenta al ser aplicado en ambos paises, en cuanto a este 
reparto de competencias que ha sido objeto de analisis por parte de la 
institucion central consiste en que mientras el Gobiemo en Espana se 
muestra mas explicito con los aspectos relacionados con la FP reglada, 
el Gobiemo de Alemania se concentra sobre la FP en el ambito de la 
empresa, pero no linicamente: la FP ocupacional y sobre todo la 
formacion continua tambien se estmcturan como responsabilidades 

gubemamentalss'*'^ . 



labia 3 

Organos regionales para la administracion, gesdony supervision 
de la educacion en Espana y Alemania 



Organos Regionales 

Semejanzas 

• Existencia de unidades poHticas descentralizadas. 

• Existencia de organismos regionales encargados de la Administracion educativa. 

• Presencia de las Camaras como organismo representativo, y algunas de sus 
competencias (asesorairiiento y supervision de empresas). 

Diferencias 

• En la denominacion y numero de las unidades politicas descentralizadas. 

• En la denominacion y numero de organismos regionales. 

• En el grado de compromise adquirido entre los organos regionales. 

En la denominacion y competencias adquiridas por las Camaras profesionales. 







Comuni dades Autonomas (CC.AA j 
(17). 


Lander (16). 


Consejerias de Educacidn. 


Ministerio de Educacion, Ciencia, 
Investigacion y Tecnologia de los 
Lander y Comites Regionales. 


Elaboracion de planes de estudio de 
FP reglada sobre los mmimos 
estipulados por el Gobiemo. 


Planes de FP sobre la escuela. 


Claras de Comercio, Industria y 
Nave gac ion. 

Participacion activa en la 

elaboracion de los planes de FP, 
formacion de responsables en las 
empresas, colaboracion con la 
programacidn, desarrollo y 


Claras de Industria y Comercio, y 
Artesanos y Oficios. 

Participacion en los planes de FP, 
competencias concretas sobre los 
examenes. 
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I evaluacion de las practicas. | | 



2.3. Organos regionales 

La descentralizacion educativa se muestra como una realidad poHtica 
que se aplica sobre ambos paises. De hecho, el esquema establecido en 
tomo al reparto de las competencias que se fijan indica que estas son 
muy semej antes, tal y como se ha desprendido del analisis realizado 
tambien sobre el texto de las Constituciones, en el primer apartado. 

Si se compara la situacion correspondiente a los Ltider con la referente 
a las Comunidades Autonomas, puede comprobarse en ambos casos 
que el proceso de delegacion de competencias propio de la 
descentralizacion se ha producido gradualmente. Ademas, existen en 
Espana Comunidades Autonomas con competencias en educacion y 
otras no, aspecto este que no sucede en Alemania, donde todos los 
Lander poseen competencias sobre educacion (y algunos generan 
incluso opciones educativas individuales y acordes a sus necesidades 
mas inmediatas). 

En cuanto a las funciones que estos han de cumplir, en Espana parece 
dominar la tarea de gestion del sistema de FP (elaboracion y 
aprobacion de planes, programas de estudio y orientaciones 
pedagogicas que complementen las ensenanzas fijadas por el Estado, 
ejecucion de programas de inversidn en programacion y equipamientos, 
aprobacion de libros de texto y material didactico que haga referenda a 
las ensenanzas complementarias, etc), mientras que en Alemania 
domina la funcion de supervision unida a la de administracion. Ambos 
completan los planes de estudio que han sido disenados a nivel mas 
centralizado, y es importante destacar como a este nivel el Ldder se 
encargara de la escuela profesional, no de la empresa (que es, como se 
ha indicado, competencia del gobiemo federal). 

Las Camaras de Comercio y Artesania en Alemania, instituciones 
autdnomas de la economia regional a las que las empresas se alian, son 
acreedoras de un mayor peso y responsabilidad que en Espana, factores 

estos que traducen todo lo anterior en competencias'^^. Una de ellas, 
que refuerza precisamente la potencia estrategica de esta institucion, es 
la que la vincula de forma directa con la organizacidn y realizacion de 
examenes profesionales parciales y finales de caracter estatal, 
formando parte integrante y activa del comit6 de los organizadores de 
los certificados. De este modo se consigue, ademas, que los examenes 
transcurran independientemente de la valoracion de las empresas, 
consiguiendo que este se desenvuelva con mayor margen de 
objetividad. Otras funciones significativas se vinculan directamente 
con el perfeccionamiento de los tfabaj adores, asi como con la 
delegacion de ciertas funciones relacionadas con decretar disposiciones 
legales sobre algunos topicos de FP (en la actuacion sobre ciertos 
colectivos, por ejemplo). 

En este sentido, y en la medida en que las Camaras profesionales van 
adquiriendo en Espana tambien mayor enjundia y son acreedoras de 
mas competencias en materia profesional, relacionadas sobre todo con 
el apartado de practicas, se ha creado en Espana el denominado 
'certificado de Camara', documento legal expedido por estas que, si 



httn7/pnaa PiHn/enaa/vQn'^S/ 



233 



10/29/01 




EL PATRON DE COOPERACION COMO GARANTIA EN LA POLITICA DE AD Page 17 of 25 



bien a efectos escolares no sustituye al certificado de FP que ha de 
concederse desde el centre educative, puede convertirse en un garante 
de calidad de cara a la consecucion de un posible puesto de trabajo en 
las empresas. 

De todos modes, lo anterior puede traducirse en una realidad segun la 
cual las Claras espanolas ya han tornado un papel mas din^ico, y su 
actividad comienza a desplegarse y a ser escuchada desde todas las 
instancias sociales responsables de la FP, en especial en el entomo 
empresarial, ya que en cierta medida se convierten en “las guardianas 

de la empresa”'^^. Desde este ambito hoy las Camaras adquieren mas 
responsabilidades: deben elaborar un censo de empresas colaboradoras, 
comprobar si los requisites que ofrece la empresa formadora satisface 
las demandas de formacion, asesorar a las empresas en materia de 
practicas y colaborar en la programacion, desarrollo y evaluacion de las 
mismas. 



Tabla4 

Instituciones de cooperacion central y regional 



Instituciones de cooperacion central y regional 

Semejanzas 

• Presencia de organismos que favorecen el acuerdo. 

• Instituciones especificas para la concertacidn social. 

• Funciones de las instituciones de cooperacion. 

Diferencias 

• Grado de especificidad sobre el nivel educativo concreto de Formacion Profesional 

• His tori a de los organismos e impacto. 

• Otras funciones. 







El Consejo General de Formacion 
Profesional. 


El BIBB y el Comite de 

Coordinacion de las Regulaciones 

Profesionales 

(Ausb il dungs o r dnunge n) . 


Organismo relativamente reciente, 
que surge en 1986. Dependiente del 
Ministerio de Trabajo y Seguridad 
Social. 


Organismo que surge con una Ley 
educativa (Ley de Pro mo cion 
Educativa de 1981). 


Asesoramiento al gobierno en 

cuestiones de Formacion Profesional 
reglada y ocupacional, y 

seguimiento de los pro gramas 
Nacionales de Formacion 

Profesional. 


Consejo al gobierno acerca de la 
Formacion Profesional; ayuda en las 
reglamentaciones profesionales y 
apoyo en la investigacion relacionada 
con Formacion Profesional. 



2.4. Instituciones de cooperacion central y regional 

Ambos paises muestran dos tipos de instituciones a traves de las cuales 
se manifiesta la cooperacion entre las diferentes instancias educativas: 
los organismos cuyas funciones consultivas y de control se extienden 

por todos los niveles educativos^^, y las que aparecen como especificas 
para el nivel particular que esta siendo objeto de estudio. Existe un 
menor niimero de instituciones basadas en la cooperacion en este 
segundo caso, concentradas en la FP. 
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Interes^donos, en efecto, por el caso mas restrictivo, en Espana, es el 
Consejo General de Formacion Profesional el que ocupa esta posicion. 
Este organismo es relativamente reciente, pero ya ha tenido un papel 
estrategico en la elaboracion de los nuevos Programas Nacionales para 
la Formacion Profesional, al encargarse, entre otros aspectos, de su 
diseno. Se espera, asi, que continue su labor en la linea comenzada, al 
tener ahora que realizar un seguimiento y evaluacion de forma 
sistematica del proceso que ya se ha desarrollado y del que ya se 
demandan las primeras valoraciones. 

En Alemania, este papel corresponde al Institute Federal de Formacion 
Profesional (3 IBB) (Bundesinsittut JnBerufsbildung). Este organismo 
profesional muestra una mayor madurez organizativa, puesto que, si 
bien ambas instituciones fueron concebidas en fechas proximas en el 
tiempo, (1986 para Espana y 1981 para Alemania), el BIBB entiende y 
aplica la cultura de la participacion social de una forma mas organica y 

practical*. De hecho, se halla conformado por un sistema de 
distribucidn de estructuras de toma de decisiones altamente complejo 
(Estado, Lader, cuerpos regionales publicos, Camaras y sindicatos 
asociaciones de empleadores), pero conjuntamente responsable de una 
FP que se establece dentro de unos parametros de calidad y exigencias 
demandados por la propia economia y sociedad alemanas. 

Como aspectos convergentes, entre los cometidos comunes que estos 
organismos realizan, se destaca la ayuda prestada para la configuracidn 
de las reglamentaciones profesionales. En Alemania se incide tambien 

en el apoyo concedido a la investigacion educativa en este terreno"*^, a 
fm de mantener constantemente actualizado el nivel con los avances 
que van produciendose en el mismo. En Espana, por su parte, se 
resaltan las funciones de asesoramiento y consulta al gobierno en 
cuestiones relacionadas con todos los niveles educativos, ademas de las 
que ya se han hecho referenda. 

3. Algunas conclusiones del estudio comparado. 

Tras el estudio comparado, a partir de la yuxtaposicion de datos (a 
traves de la cual se ha podido conocer cuales son las semejanzas y las 
diferencias establecidas en tomo a una serie de tematicas), es el 
momento de retomar el objetivo inicial y poder estudiar los efectos que 
dicha premisa ejerce sobre la realidad de la FP en Alemania y en 
Espana. En efecto, la hip6tesis que bien podria formularse como 'El 
patron de cooperacion entre las diferentes instancias en la FP espanola 
asegura una FP de calidad en la medida en que se produce un reparto de 
competencias efectivo y regulado, como o.curre en el modelo alem^' 
resulta confirmada, a partir de los resultados obtenidos en la 
Comparacidn, y puede concluirse con la afirmacion de que existe, a un 
nivel generalizado, una firme consolidacipn en la tendencia que 
persigue el fortalecimiento de patrones de cooperacion educativa entre 
las diferentes instancias responsables de la FP, desde los niveles mas 
centrales a los niveles mas regionales, pasando por las estructuras de 
concertacion, que tambien ejercen un peso notable a la hora de 
establecer decisiones en materia profesional. 

Las consecuencias que la democratizacion ejerce sobre todos los 
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aspectos de la realidad educativa parecen traducirse paradojicamente de 
forma mas legitima en Espana que en Alemania, ya que es en este 
primer pais donde se produce una distribucion de competencias entre 
centro y periferia mas proporcional a nivel juridico, politico y 
administrativo. En Alemania tiene lugar un reparto taxativo entre las 
dos maneras de entender la FP: la FP en la empresa, y la FP en la 
escuela, ambas condicionadas y determinadas por la estructura federal 
del Estado. Desde la Constitucion ya puede vislumbrarse este aspecto, 
al juzgar el papel concedido tanto al Estado como a las competencias 
que van a ser compartidas por otras instancias. 

De lo anterior puede desprenderse ademas que el nivel de 

responsabilidad parece ser de mayor enjundia en los Lader^^ ya que al 
nombrar las competencias compartidas en la legislacion, en Espana se 
habla de que “las Comunidades Autonomas podran asumir 

competencias Ahora bien, los articulos espanoles que asignan 

competencias, como es el caso del presente, estan llenos de frases cuya 
retorica es dificil de desentranar (incluso cuesta encontrar en ellas el 
apartado concrete referido a educacion), de modo que las transferencias 
se suceden muchas veces “con la misma facilidad con que se dan 
portazos, y este tipo de normas suelen traer mas conflictos que 

soluciones”^^. Asi, dentro de la Constitucion, las Comunidades 
Autonomas completaran las ensenanzas minimas que el Estado fija. La 
Ley Fundamental alemana es mas explicita en sus intenciones de 
acarrear responsabilidades para las regulaciones de FP en la empresa, 
en la Formacion continua y en la participacion en la legislacion y 
administracion de la Federacion y asuntos de la Union Europea, 
mostrando con ello un caracter de nuevo mas independiente y a la vez 

mas responsable con este nivel educativo concrete . Sera 
precisamente en los referentes de la legislacion sobre FP donde se 
completen estos aspectos. 

En ambos paises, la figura del gobiemo resulta necesaria como eje articulador de la educac 
Ahora bien, en Alemania existe un concepto de FP mucho mas vasto e integrador que el q 

En cuanto a los organos regionales, yde acuerdo con los principios que rigen el principio d 

Volviendo al principio de descentralizacion, hoy se reclama mas que nunca este razonamie 
Si se traslada lo arriba indicado al caso concrete que esta siendo objeto de estudio y compa 
El Ministerio de Educacion, Ciencia y Tecnologia que corresponde a cada uno de los Liide 
Da la impresion de que los organos regionales van a mostrarse mas sensibles a las necesid 
Un elemento que puede ayudar a la reflexion establecida en tomo a las competencias reale 
En cuanto a las instituciones de cooperacion central y regional, en Alemania se ha podido 
En Espana, la atencion y consiguiente interes aplicados sobre la FP, aspectos estos que im 
Finalmente, tan solo queda indicar que las formas de cooperacion no se agoteui en ambos p 
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Para concluir, solo queda insistir en que el presente estudio se eiunarca dentro de los analis 

Notas 

* DE PUELLES BENITEZ, Manuel: “Educacion y autonomia en el modelo espanol de de 

^GARCIA GARRIDO, Jose Luis: Fundamentos de Educacio Comparada. Madrid. Dykins 

^La diferencia en las competencias (educativas, en este caso) entre unas Comunidades Aut 
Teoni de 
la Educacio. 

PoHcay 

Educacio. pp., 556-557. 

^Esta deberia cumplir un papel principal. Puelles entiende que no puede seguir teniendo la 

^ Descrita detalladamente en EURYDICE: Um 
deada 

de reformas en la educacio obligatoria de la Unia Europea (1984-1994). Op. Cit. pp., 125 

^DE PUELLES BENITEZ, Manuel: “Educacion y autonomia en el modelo espanol de des 

^Tal y como senala CABALLERO CORTES, Angela: “Educacion y Formacion Profesion 

^En este sentido, las Comunidades Autonomas que poseen un mayor campo competencial 

^Como indica JATO SEIJAS, Elisa: La Formacio Profesional en el contexto europeo: nue 
desafis 

y tendencias. Barcelona. Estel,1998. p., 241. 

es que, tal y como comenta Jimenez, “el modelo tiene un fuerte origen centralista del 
’ 'Su origen se situa en 1984, con motivo del Acuerdo Economico y social, pacto tripartito 
'^La FP ocupacional que no se realiza en los centros de formacidn es responsabilidad del 
*^DE PUELLES BENITEZ, Manuel: “Educacidn y autonomia en el modelo espanol de de 
*"^E1 Liider puede ser defmido como uno de los estados constitutivos de la Republica Fede 
'^CASCAJO CASTRO, Jose Luis y GARCIA ALVAREZ, Manuel: Constituciones extran 
’^GRUPO DE TRABAJO “INFORME EDUCATIVO” DEL INSTITUTO MAX-PLANC 
'^Ibidem, p., 193. Se anade que “el Canciller Federal dirigira los asuntos gubemamentales 
'^Asi, como expresa y matiza Richter, la Constitucion no provee expresamente al Gobiem 
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^^Al respecto de las competencias establecidas desde el Ministerio, segun De Puelles, esta 

90 

Estos Ministerios son los que a continuacion se citan: el Ministerio de Asuntos Extranje 
^'Especificadas en su Articulo 73. CASCAJO CASTRO, Jose Luis y GARCIA ALVARE 

99 

CEDEFOP Y EURYDICE: Estructuras de los sistemas educativos y de formacia inicial 

9 ’^ 

Tal y como lo estipula el “Protocolo Conjunto relativo al Establecimiento de Objetivos 
deada 

de reformas en la educacia obligatoria de la Unia Europea (1984-1994). Op. Cit. p., 101. 
^'^TAIVITIAN, Roland: “El dialogo social comunitario: evolucion y desafios”. Formacia 
WEEGMANN, Ingeborg: “El papel del Estado y de los interlocutores sociales en la FP”. 
En Capitulo II Seccion 6 'Establecimiento y Tareas".BUNDESMINISTERIUM FUR BI 
^^En estrecha colaboracion con otra institucion, la Deutsches InstitutfnFernstudienforsch 
Asi, por ejemplo, en 1986, y a tenor de la financiacion del sistema de FP, la Ley obligab 

90 

Como es el caso del programa LEONARDO DA VINCI, programa de intercambio de est 

^^KOTCH, Richard y REULING, Jochen: “The responsiveness and regulation of Training 

^ En su Reglamento aparece como funcion el tratamiento y solucidn de “Materias de rele 
deada 

de reformas en la educacia obligatoria de la Unia Europea (1984-1994). Op. Cit. pp., 101 
^^Una medida que se tomo en concrete por parte de la Conferencia fue, en el “Acuerdo ma 
^^IV Parte 'Comites de Formacion ProfesionaF. Division I y II Seccia 55 (Deberes). BUN 
^^Para un conocimiento mas exhaustive del impacto de la Constitucion sobre la educacidn 
^^GENERALIDAD VALENCIANA: La Constitucia espabla. Op. Cit. pp., 1 y 36-37. 
^^Ibidem. Articulo 40. p., 10. 

^^Ibidem. Articulo 148. p., 36. 

^^Ibidem. Articulo 149. p., 37. 

^^CASCAJO CASTRO, Jose Luis y GARCIA ALVAREZ, Manuel: Constituciones extran 
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^^Ibidem. Articulo 50. pp., 189-190. 

^hbidem. Articulo 63. p., 192. 

^^Este aspecto puede ser consultado en RICHTER, Ingo: Chapter three: “West Germany, 
deada 

de reformas en la educacia obligatoria de la Unia Europea (1984-1994). Bruselas. EURY 

^^Una informacion mas exhaustiva sobre los objetivos y el espiritu que rige al citado progr 
poHca 

de Formacia Profesional (1997-2000). Madrid. FORCEM, 1996; MINISTERIO DE TRA 

'^'^Ademas, e incidiendo sobre este aspecto, no deja de sorprender como se hace referencia 

^^Para revisar el papel y funciones acometidas por las Camaras en Alemania, consultese:K 

^^Para revisar el papel y funciones acometidas por las Ctoiaras en Espana: CONSEJO SU 

^^Estas funciones son cubiertas por la Conferencia Permanente de Ministros de Educacion 

Establecido a partir del Capitulo II Seccion 6 'Establecimiento y Tareas '.BUNDESMIN 

^^En Alemania, establecido a partir de las directrices de la Ley de Promocion de la Forma 

^®Como ya se senala en la Ley Fundamental, la caracteristica mas singular que describe es 

^'Tal y como se desprende del Capitulo Tercero, 'De las Comunidades Autonomas', en el 

^^Como apostilla PANIAGUA, Juan Luis: Espaa, 1978: una Constitucia para el pueblo. 

^^Tal y como lo estipula la Ley Fundamental en su Articulo 73. CASCAJO CASTRO, Jos 

PEDRO, Francesc: 

Potica 

educativa, 

poUques 

educatives. Barcelona. Universitat Obertade Catalunya, 1997. 



Quiza sea interesante recordar aqui que el concepto de 'descentralizacidn' no se agota e 
potica y 

legislacia educativas. Capitulo Septimo: 'Descentralizacion educativa'. Madrid. Sintesis, 

^^Tal y como indica Delors en DELORS, Jacques: “Presentacion del Informe a la UNESC 
Desafis 

y oportunidades. Documentos de un debate. Madrid. Fundacibn Santillana, 1997. p., 44. 
^■^FERNANDEZ SORIA, Juan Manuel: Manual de 
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poHcay 

legislacia educativas. Op. Cit. p., 171. 

^^Este estudio se enmarca en la Tesis Doctoral defendida por la autora, y que lleva como ti 
poHcas 
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Abstract 

Chief Academic Officer (CAO) is the most common 
position title before assuming the presidency of a college 
or university. Results from a national survey are used to 
develop a profile of the CAO in each respective Carnegie 
institutional classification. The typical CAO in four-year 
institutions is Caucasian, male, 54 years old, and married. 

He holds a doctoral degree, most likely in humanities or 
social sciences, and has held the CAO position for 5 or 
fewer years. Most often, the CAO served as a Dean or 
Associate Dean in the previous position. All CAOs have 
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classroom experience, but 3% have never taught full-time. 
With only slight variances among the percentages, these 
characteristics are similar for each of the respective 
Carnegie classifications. Comparisons are also made 
between the characteristics of presidents and CAOs. 



Introduction and Background 

A critical need of any organization is leadership (Martin & 
Strauss, 1956). There are a number of titles common to the position 
that provides academic leadership in colleges and universities, Provost 
or Vice President for Academic Affairs are common examples. In this 
article we use the term chief academic officer (CAO) to refer to all 
individuals who have overall responsibility for the academic 
component of an institution of higher education. The simple fact that 
the chief academic officer (CAO) has authority and influence over 
both the goals and objectives and the resources dedicated to the 
instructional program of a college or university points to the overall 
importance of this leadership position (Weingartner, 1996). Given the 
current state of declining resources and eroding public confidence, 
effective leadership of the academic program has become a key 
challenge facing higher education organizations (Martin & Samels, 
1997). The challenge is so great that Birnbaum (1992) announced that 
in many instances the CAOs impact on an institution was as great, or 
even greater, than that of the president. 

Who are the people primarily responsible for providing 
academic leadership in higher education institutions? Given the 
importeince of the role, it is interesting that so little attention has been 
paid to them. Since 1980, only six studies of individuals in the CAO 
position have been reported in the literature. Three studies reported 
information on chief academic officers in two-year colleges 
(Hawthorne, 1994; Twombly, 1988; Vaughan, 1990), two studies 
included individuals at both two- and-four-year institutions (Moden, 
Miller, & Williford, 1987; Warner, Brazzell, Allen, Bostick, & Marin, 
1988) and one study was limited to CAOs in four-year institutions 
(Moore, 1983). As this investigation focuses on CAOs at four-year 
institutions, only applicable previous research is included to provide a 
background. 

Moden, Miller, and Williford (1987) developed a stratified 
random sample based on the student FTE size of 3,328 higher 
education institutions and their branches. Of the 415 institutions 
surveyed, usable returns were received from 331 (73%). Two-year 
institutions employed 40% of the respondents. Slightly more than 
four-fifths (81%) of the positions were held by males. The ages of the 
~ ■ ■ CAOs ranged fi’om 34 to 67, with a mean of 49 years. Slightly less 

then one-fourth (22%) of the CAOs had been in the position for one 
year or less and 35% reported 5 or more years in office. 

Warner et al. (1988) surveyed a randomly selected sample of 
800 administrators at the level of dean or above. The sample was not 
restricted by institutional type, with surveys sent to universities, 
colleges, community colleges, and technical schools. A usable 
response rate of 49% was realized. Of those responding to the query of 
title of current position, 41 (11%) were CAOs. Results of the survey, 
however, are presented for all administrative positions, ranging from 
assistant or associate dean to president and chancellor. 
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Moore (1983) surveyed a stratified random sample of 4,000 line 
administrators representing 1 ,600 accredited four-year institutions. 
Responses were received from 2,896 (73%) administrators in 55 
positions. Of the respondents, 151 (5%) were CAOs. The vast majority 
of CAOs were male (86%), Caucasian (96%), and married (83%). The 
ages of the CAOs ranged from 37 to 68, with the majority (51%) 
between the ages of 45 and 55. Only 14% of the CAOs had been in the 
position for 1 1 or more years, with 59% reporting a tenure of 6 or 
fewer years. Almost all (99%) of the CAOs had earned a doctoral 
degree. A vast majority (88%) held academic rank, more than three- 
fourths (78.6%) were professors, and a majority (60%) were tenured. 

The American Council on Education (ACE) has presented three 
profiles of the career experiences of presidents from data gathered in 
1986, 1990, and 1995. Each of these profiles revealed that chief 
academic officer was the most common position title before assuming 
the presidency. In the most recent report (Ross & Green, 1998), CAO 
was the previous position of 26.5% of the respondents, followed by 
president at another institution (19.9%), and deans or their associates 
(11.9%). 

Using data provided by the National Center for Educational 
Statistics for the years 1974-81, Rickard (1982) reported that CAOs 
had the highest rate of turnover of top level administrative offices 
(20%). This trend has not changed as the CAOs experienced an annual 
turnover rate of 19% for the years 1985-92, again a rate higher than 
chief business officers, chief student affairs officers, and presidents 
(Mooney, 1993). 

Considering the role of the position in the career experiences of 
presidents, the rate of turnover by position holders, and the importance 
of the position to higher education organizations, the CAO position 
emerges as the “next step” in understanding career paths in higher 
education administration. The purpose of this article, therefore, is to 
add to the research on administrative careers in higher education by 
developing a profile of chief academic officers at four-year colleges 
and universities. 

Methodology 

Survey Instrument 

We contacted Marlene Ross, principal author of the ACE 
reports, who granted us permission to adapt the ACE President’s 
Survey to gather data regarding chief academic officers (M. R. Ross, 
personal communication, October, 1 997). There were three reasons we 
selected the ACE instrument as a base for our inquiry. First, the three 
presidential profiles are the most comprehensive data concerning 
administrative careers in higher education. Second, similarity in 
instrumentation would allow for comparisons between the experiences 
of chief academic officers and presidents. Third, we hoped to 
encourage other researchers to take a similar approach in examining 
other top-level positions in higher education. 

Our revised survey instrument, therefore, is based on the same 
demographic and career experience questions as found on the ACE 
presidential survey with two modifications. First, we asked for the 
specific position title. Ross and Green (1990) stressed that beyond the 
general agreement that president or chancellor indicates the chief 
executive officer, there is little consensus concerning the specific 
responsibilities associated with administrative titles in higher 
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education. Using data from the 1995 Higher Education Directory, 
Martin and Samels (1997) found that the words vice-president and 
dean each occurred in the chief academic officer title of approximately 
40% of the reporting institutions, with provost listed as the title of 
approximately 1 6% of the reporting institutions. Second, we were 
interested in the faculty experiences of the CAOs. One measure of 
connection to the academic component of the institution is whether or 
not the CAO holds faculty rank or tenure. While some institutions do 
not offer rank or tenure to administrators, the practice is still followed 
in many instances. Further, holding rank or tenure in the previous 
position and the highest faculty rank achieved also provide insight to 
the academic connection. To gather information on faculty 
experiences, we asked about rank and tenure for the current and two 
previous positions. In addition we questions on the highest rank 
achieved and the total years of full-time faculty experience. The 
revised survey instrument was piloted to eight chief academic officers, 
representing the respective Carnegie classifications (1994). 

Population, Survey Method, and Response 

The survey was mailed in November of 1997 to the Chief 
Academic Officer at 1372 four-year colleges and universities. This 
population included all institutions listed in the 1994 Carnegie 
classifications of higher education, limited to accredited institutions as 
listed in the 1 997 Higher Education Directory. An initial follow-up 
survey was sent in January of 1998. Finally, follow-up by fax and 
telephone was conducted during May of 1998. Overall, 1058 surveys 
(77%) were returned. After accounting for positions that were vacant 
or currently filled by individuals with the title of acting or interim and 
eliminating responses that did not come from the chief academic 
officer, 971 usable surveys (71%) were returned from the population. 
The usable rate for the respective classifications ranged from a low of 
51% (Doctoral Universities I) to a high of 78% (Baccalaureate 
Colleges I). Table 1 presents information regarding the usable return 
rate. 

Table 1 

Usable Returns by Carnegie Classification 



Classification 


N 


Return 


% 


Research Universities I 


88 


56 


64 


Research Universities II 


37 


23 


62 


Doctoral Universities I 


51 


26 


51 


Doctoral Universities II 


58 


42 


72 


Master’s Colleges and Universities I 


430 


305 


71 


Master’s Colleges and Universities II 


89 


68 


76 


Baccalaureate Colleges I 


165 


128 


78 


Baccalaureate Colleges II 


454 


323 


71 


Total 


1372 


971 


71 



Data analysis 
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We created eight databases for each respective Carnegie 
classification. Where possible, responses were coded numerically and a 
written guide of coded categories was created for reference. In an additional 
effort to assure reliability, we hired individuals independent of the study to 
enter the data and additional independent observers to substantiate the 
databases. For the Carnegie classifications with fewer than 150 responses 
(RI, RII, DI, DII, Mil, BI) the observers verified all survey information to 
the databases. For the Carnegie classifications with responses above 300 
(MI, BII) the observers selected a random sample of 50% of the surveys to 
compare to the databases. Overall 658 (68%) of the surveys were examined, 
with errors in the database identified for 12 instruments. This resulted in a 
98% reliability rating for the data. 

Personal Characteristics 

Information on the characteristics of sex, race, age and martial status 
is presented in Table 2. The characteristics of spousal employment and 
religious affiliation appear in Figures 1 and 2, respectively. 

Table 2 

Demographic Profile by Carnegie Classification 



Characteristics 


RI 


RII 


DI 


DII 


MI 


Mil 


BI 


BII 


Sex 

(percentage) 


N=50 


N=23 


N=27 


N=43 


N=299 


N=70 


N=126 


N=318 


Male 


78 


87 


67 


88 


75 


73 


71 


74 


Female 


22 


13 


33 


12 


25 


27 


29 


26 


Race/Ethinicity 

(percentage) 


N=51 


N=23 


N=27 


N=43 


N-296 


N=70 


N=128 


N=320 


Asian 


— 


— 


— 


— 


— 


3 


1 


1 


African- 

American 


6 


— 


4 


5 


7 


6 


4 


6 


Caucasian 


94 


100 


96 


95 


90 


90 


95 


91 


Hispanic (non- 
black) 


— 


— 


— 


— 


1 


— 


— 


1 


American Indian 


— 


— 


— 


— 


>1 


— 


— 


— 


Multiracial 


— 


— 


— 


— 


>1 


— 


— 


1 


Other 


— 


” 


— 


— 


>1 


-- 


— 


— 


Age (years) 


N=48 


N=22 


N=25 


N=41 


N=279 


. N=67 


N=119 


N=318 


Mean 


55 


56 


64 


56 


54 


55 


52 


53 


Median 


55 


56 


56 


56 


54 


55 


52 


53 


Mode 


55 


57 


56 


56 


54 


55 


53 


53 


Range 


43-66 


46-70 


41-63 


45-70 


37-68 


41-68 


37-67 


34-73 


Marital Status 
(percentage) 


N=50 


N=23 


N=27 


N=43 


N=298 


N=70 


N=129 


N=323 


Never married 


2 


— 


7 


2 


6 


6 


6 


6 


Religious Order 


— 


— 


— 


2 


2 


6 


— 


5 
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Married 


96 


96 


59 


86 


84 


81 


86 


80 


Separated 


— 


— 


— 


— 


1 


1 


2 


1 


Divorced 


2 


4 


27 


10 


6 


6 


5 


7 


Widower/Widow 


— 


— 


7 


— 


1 


— 


1 


1 



Not 

em^^d 



Employed 

outside 

higher 

education 

8% 



Employed at 
same 
institution 




another 

institution 

49% 



Figure 1. Employment of Chief Academic Officers Spouses 



Jewish 

Other 6% 




9% Protestant 

17% 



Figure 2. Religious Affiliation of Chief Academic Officers 
Sex and race 

Twenty-five percent of all CAO respondents were women. As 
indicated in Table 1, the representation of women in the CAO position 
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ranged from a high of 33% in Doctoral I institutions to a low of 12% in 
Doctoral II institutions. Members of minority groups held 8% of the CAO 
positions. African-American CAOs constitute the largest minority group 
(5.8%), followed by Asian and Hispanic (.6% respectively), multiracial 
(.3%), and American Indian (.1%). Members of minority groups are most 
represented in the CAO position at MI and Mil institutions. No respondents 
from RII institutions indicated that they were members of minority groups. 
Almost one-third (32%) of the minority respondents were female; 44% of 
African American respondents were female. 

Age 



The median and, after rounding, the mean age of the CAOs was 54 
years. Both the youngest (34 years) and the oldest (73 years) respondents 
were at BII institutions. Slightly more than two-thirds (70%) of the CAOs 
were between the ages of 40 and 56. Among women, 73% were between the 
ages of 40 to 56. The mean age of women CAOs is lower than their male 
counterparts at research institutions (50 to 55 at R-I and 48 to 56 at R-II) 
and higher than the male CAOs at M-II institutions (59 to 55). Only 1% of 
all respondents were below the age of 40 and no respondents fi’om Research 
(I and II), Doctoral (I and II), or Masters II institutions indicated they were 
less than 40 years of age. In terms of age, the responses of minority 
members were similar to the population as a whole. The mean age of 
minorities was 53 and 70% were 56 years old or younger. 

Marital status, spousal employment, religious affiliation 

The vast majority of CAOs are married (83%), ranging from a high of 
96% in the RI and RII categories to a low of 59% in the DI category. 

Slightly more than 8% of the CAOs have never been married, 2.7% of these 
indicating they were members of religious orders. Among married CAOs, 
76% had spouses who were employed. Almost two-thirds (64.5%) of the 
working spouses were employed in higher education, 17.5% at the same 
institution as the CAO. Virtually two-thirds (65.4%) of the spouses were 
employed on a full-time basis. Spouses of Mil CAOs were most likely to 
work (83%) and were most likely to be employed in higher education 
(78%). Spouses of RII CAOs were least likely to work (50%). Slightly more 
than one-half (51%) of the CAOs identified themselves as Baptist, 

Episcopal, Methodist, Presbyterian or other type of Protestant; 24% were 
Catholie; 6% were Jewish, and 1% listed themselves as Eastern Orthodox. 
Among the CAOs who reported memberships in religious orders, 38% were 
ordained ministers, 32% were Catholic sisters, and 26% were Catholic 
priests or brothers. 

Professional Characteristics 
Position Title 

Table 3 presents words most often reported in the titles of chief 
academic officers. In order to develop these categories, specific adjectives 
such as senior, executive, academic, and instructional were removed. Vice 
President (32%) is the most common title of the CAO, followed by Vice 
President and Dean (17%), Vice President and Provost (16%), Provost (12%), 
and Dean (1 1%). Vice President, Vice President and Provost, and Provost are 
the only titles found across all of the respective Carnegie classifications. 
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Table 3 

Generic Titles of Chief Academic Officer 
(percentage by Carnegie Classification) 



Title 


RI 


RII 


DI 


DII 


MI 


Mil 


BI 


BII 


Vice Chancellor 


12.5 


— 


— 


2.4 


4.6 


4.4 


1.6 


2.5 


Vice Chancellor & 
Provost 


10.7 


8.7 


— 


2.4 


5.2 


2.9 


— 


— 


Vice President 


5.4 


8.7 


23.1 


26.2 


40.7 


39.7 


8.6 


40.2 


Vice President & 
Provost 


41.1 


60.9 


30.8 


42.9 


25.6 


5.9 


5.5 


6.2 


Vice President & 
Dean 


— 


— 


3.8 


2.4 


4.9 


25.0 


37.5 


26.0 


Provost 


30.4 


17.4 


30.8 


16.7 


11.5 


7.4 


14.8 


6.5 


Provost & Dean 


— 


— 


7.7 


2.4 


1.3 


4.4 


7.8 


3.1 


Dean 


— 


4.3 


— 


4.8 


5.2 


10.3 


23.4 


15.5 


Other 


— 


— 


3.8 


— 


1.0 


1.5 


0.8 


— 



Academic Background 

The field of study of CAOs is presented in Table 4. Overall, more 
CAOs studied humanities/fine arts (30%), followed by social sciences (28%), 
education (15%), and physical/natural sciences (12%). Social sciences 
emerged as the predominant field of study for four of the respective 
classifications (RI, DI, DII, BII). Humanities/fine arts were the predominant 
field of study for three classifications (MI, Mil, BI) and physical/natural 
sciences was the predominant field in the remaining classification (RII). The 
Ph.D. was earned by 86% of the CAOs, 9% had been awarded the Ed.D., 3% 
held professional degrees, and 2% reported the master’s as the highest 
awarded degree. 



Table 4 

CAO Field of Study 
(percentage by Carnegie Classification) 



Title 


RI 


RII 


DI 


DII 


MI 


Mil 


BI 


BII 


Agriculture 


3.6 


— 


— 


2.4 


— 


— 


— 


— 


Biological 

Sciences 


. 9.1 


8.7 




4.8 


4.7 


6.0 


3.1 


4.9 


Education 


5.5 


4.3 


8.0 


11.9 


15.8 


14.9 


3.9 


21.4 


Engineering 


9.1 


4.3 


4.0 


11.9 


0.7 


— 


— 


— 


Health Professions 


3.6 


— 


— 


4.8 


0.4 


3.0 


— 


— 


Medicine 


3.6 


4.3 


— 


2.4 


— 


— 


— 


— 


Humanities/F ine 
Arts 


14.5 


4.3 


32.0 


26.2 


29.9 


35.8 


40.9 


29.4 


Religion/Theology 


1.8 


— 


— 


— 


4.0 


4.5 


3.1 


4.5 
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Physical/Natural 

Sciences 



20.0, 47.8, 12.0, 4.8, 14.4, 13.4, 7.9, 7.8 



Social Sciences 



25.5 



26.1 



44.0 



30.8 



25.8 



19.4 



40.2 



29.8 



Law 



3.6 



0.4 



Other 



4.0 



3.0 



0.9 



2.3 



Rank and Tenure 



Information concerning faculty rank and tenure is presented in Table 5. 
It is more common for CAOs to hold rank than to hold tenure. Of the CAOs 
responding to this query, 89% held faculty rank with 64% also holding 
tenure. This difference comes primarily from the Mil and BII classifications, 
each with more than a 35% difference between the number of CAOs holding 
rank and the number holding tenure. Full professor is the most common rank, 
reported by 73% of the CAOs. 

In the immediate prior position, the same percentage held faculty rank 
(89%), but a greater percentage (70%) also held tenure. Again, the greatest 
difference in numbers holding rank and numbers holding tenure are in the 
Mil and BII classifications. Full professor was the rank held by 63% of the 
respondents. In response to the question about highest faculty rank ever held, 
74% reported full professor, 20% reported Associate Professor, and 5% 
reported Assistant Professor. 



Table 5 

Rank and Tenure Characteristics 
(percentage by Carnegie Classification) 



Characteristic 


RI 


RH 


DI 


DII 


MI 


Mil 


BI 


BII 


CAO Position 


N-52 


N=23 


N=27 


N=43 


N=303 


N=70 


N=127 


N=319 


Hold Tenure 


98.1 


100 


85.2 


93.0 


66.3 


52.9 


72.4 


47.0 


Hold Rank 


98.1 


100 


92.6 


100 


87.1 


90.0 


96.1 


84.0 


1st Prior Position 


N=52 


N=22 


N-26 


N=42 


N=302 


N=70 


N=128 


N=313 


Hold Tenure 


98.1 


95.5 


88.5 


85.7 


74.2 


58.6 


76.6 


55.6 


Hold Rank 


98.1 


100 


92.3 


92.9 


87.4 


91.4 


95.3 


83.1 


2nd Prior 
Position 


N=49 


N=23 


N=25 


N=41 


N=282 


N=67 


N=99 


N=276 


Hold Tenure 


95.9 


100 


96.0 


90.2 


88.7 


85.1 


90.9 


80.0 


Hold Rank 


98.0 


91.3 


88.0 


70.7 


78.0 


62.7 


68.7 


55.8 


Highest Rank 
Held 


N=51 


N=22 


N=26 


N=41 


N=286 


N=64 


N-118 


N=279 


Professor 


100 


100 


88.5 


87.8 


76.9 


75.0 


74.6 


60.6 


Associate 

Professor 


— 


— 


7.7 


9.8 


19.2 


21.9 


20.3 


29.0 


Assistant 

Professor 


““ 


— 


— 


2.4 


3.1 


3.1 


3.4 


10.0 


Instructor 


— 


— 


— 


— 


0.4 


-- 


1.7 


0.4 


Lecturer 


— 


— 


3.8 


— 


— 


— 


-- 


— 


Emeritus 


— 


— 


— 


— 


0.4 


— 


— 


-- 
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I I I I I I I I I 

Years in Positions 

/ 

As shown in Table 6, there were new CAOs in six of the eight classifications. In four classificati 
The length of time spent in the two prior positions is also shown in Table 6. As with the CAO po 
Table 6 also reveals differences in the CAOs full-time teaching experience across the respective 



* Multiple modes 

Career Paths 

Tables 7 and 8 present the title of the first and second previous position. Dean is the most comm 
Lateral movement^ from CAO at one institution to the same position at another institution, was f 
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. ^Positions grouped as University Administration includes assistant to president, assistant to chancellor, and director of ins 
**Positions grouped as College/School Administration includes director of graduate studies (for a specific college or schoo 
♦ ♦♦Positions grouped as Unit Administration include chair, director, coordinator, or head of a department or program. 



^Positions grouped as University Administration include assistant to president, assistant to chancellor, and director of instit 
**Positions grouped as College/School Administration include director of graduate studies (for a specific college or school 
♦ ♦♦Positions grouped as Unit Administration include chair, director, coordinator, or head of a department or program. 

As shown in Table 9, slightly more than the majority of CAOs (53%) were internal candidates fo 
The vast majority of CAOs (88%) stayed within the respective Carnegie classifications in movin 
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Discussion and Conclusions 

Developing a profile of the CAO was the primary purpose of this study. The typical CAO in four 
As mentioned earlier, one of our purposes in adapting the ACE Presidential Survey was to allow 
In 1995, females constituted 17.2% of the presidents at four-year institutions. Their largest repre 
The representation of women in faculty and administrative positions has been a concern of highe 
Minorities represented 10.3% of four-year college presidents in 1995. Their largest representatio 
The representation of minority groups in faculty and administrative positions has also been a hig 
A difference in the employment patterns of CAO and president spouses was also identified. The 
There are three key differences in the professional characteristics of presidents and CAOs. One d 
A second important difference between presidents and CAOs is in their movement into the positi 
The greatest difference in professional characteristics, however, was faculty experience. Slightly 
Our final reason for adapting the ACE Presidential Survey was to encourage a similar approach i 
Leadership in higher education continues to be an important topic. Developing profiles of indivi 
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Abstract 

Charter schools involve a trading of autonomy for 
accountability. This accountability comes through two 
forces — markets through the choices of parents and 
students, and accountability to government through the 
writing of contracts that must be renewed for schools to 
continue to operate. Charter schools are supposed to be 
more accountable for educational performance than 
traditional public schools because authorizers have the 
ability to revoke charter contracts. Here, I focus on one 
central component of accountability to government: 
performance accountability or accountability for 
educational outcomes to charter school authorizers 
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through the revocation or non-renewal of charter 
contracts. In this paper, I suggest that contract-based 
accountability for educational performance in charter 
schools may not be working as proponents argued it 
would. This article explores some explanations for why 
there are very few examples of charter schools that have 
been closed primarily because of failure to demonstrate 
educational performance or improvement. Future work 
will need to test if these challenges for authorizers hold in 
a variety of contexts. The conclusion examines the 
implications of these findings for the future of charter 
school accountability. 



Introduction 

Charter schools are premised on the idea that one can “trade” 
autonomy for accountability — specifically, that if one provides greater 
autonomy to individual schools, through deregulation and/or school- 
site control over finances, hiring, curriculum and mission, then one can 
place greater demands on the educational performance produced by 
those schools (Kolderie, 1 990; Nathan, 1 996) According to one 
analysis of charter school legislation, “[m]ost charter school statutes 
with sections on legislative intent are quite explicit in their expression 
of the legislature’s demand for accountability for student 
performance” (Millot, 1996, p. 9). This accountability comes through 
two forces — markets through the choices of parents and students, and 
accountability to government through the writing of short-term 
(generally 3-5 years) contracts that must be renewed in order for 
schools to continue to operate. (Note 1) 

While the theory underlying the charter school idea varies 
somewhat from state to state, a central part of that theory is that charter 
schools will be more accountable for educational performance than 
traditional public schools, largely because authorizers have the ability 
to revoke or not renew charter contracts (Kolderie, 1990; Nathan, 
1996). This article focuses on one central component of accountability 
to government — that of performance accountability or accountability 
for educational outcomes to charter school authorizers (the public 
entities that grant operators charter contracts) through the revocation 
or non-renewal of charter contracts. 

The early information - and this is early information, as only 
29% of states with charter schools have had schools go through the 
renewal process - is that “in those states [where schools have come up 
for renewal], almost all schools seeking renewal have been 
successful” (SRI International, 2000, p. 56). While a number of 
schools have closed (39 as of 1999, according to the Center for 
Education Reform), these closures have mostly been for organizational 
or financial reasons; as Finn and his colleagues argue, the most 
common reasons for closure have been, “organizational chaos, 
management meltdown, and fiscal shenanigans” (Finn, Maimo, & 
Vanourek, 2000, p. 137). One explanation for the continuing operation 
of the vast majority of charter schools is that they are, indeed, 
producing improved student achievement. However, reseeirch and 
evaluations suggest a more complicated story, with a mix of success 
and struggles (Horn & Miron, 1998; Horn & Miron, 1999; Public 
Sector Consultants & MAXIMUS, 1999; RPP International, 1999). 
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In this article, I suggest that contract-based accountability for 
educational performance in charter schools may not be working as 
proponents argued it would. According to SRI International, “[f]ew 
charter school authorizers have revoked or not renewed charters 
because of student performance problems” (SRI International, 2000, p. 
57). This article explores some explanations for why there are very 
few examples of charter schools that have been closed primarily 
because of failure to demonstrate educational performance or 
improvement. Possible explanations, including the challenges of 
determining school quality, the strong and vocal support of charter 
school communities (relative to the quiet and diffuse public interest), 
and concerns about damaging the charter school movement, provide 
strong incentives for authorizers to allow schools to continue to 
operate. I am not suggesting that no authorizers are taking 
performance accountability seriously — some clearly are - but that 
acting as the originators of charter schools intended can be very 
difficult for authorizers. 

Following a brief description of the data sources drawn on, I 
discuss the accountability ideal for charter schools, and the 
relationship between this ideal and ideas of a “new accountability” for 
public education more generally. This is followed by a description of 
how authorizers are addressing their different roles that relate to 
accountability - including approving applications, overseeing schools, 
and granting new contracts. In the next section, I explore some 
possible explanations for why charters schools are rarely closed and 
some of the “middle grounds” authorizers have created to provide 
incentives and sanctions to schools without actually forcing them to 
close. Future work will need to test if these challenges for authorizers 
hold in a variety of contexts. The conclusion examines the 
implications of these findings for the future of charter school 
accountability. 

The Accountability Ideal 

Charter school accountability has both unique components, 
especially the granting of an actual charter contract allowing an entity 
not governed directly by a school board to operate a public school, and 
facets that are closely intertwined with broader changes in ideas about 
public school accountability. In the following two sections, I examine 
both of these aspects of accountability. 

Ideals of accountability for charter schools 

One of the challenges of talking about accountability for charter 
schools — or a host of other issues — is the variation among states as to 
the interpretation of the charter school idea in legislation (Buechler, 
1996 (July); Bulkley, 1999c; Mulholland, 1996). Accountability is 
often separated into two components - to whom an entity (such as a 
school) is accountable, and for what they are accountable (Elmore, 
1995). As noted earlier, accountability for charter schools has two 
facets: 



• Market-based accountability, which operates through the 
choices of parents and students; and 

• Performance-based accountability, which operates through 
contracts between charter schools and their authorizers 
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specifying the educational and other outcomes the school will 
produce if it is to continue to operate. 

The reliance on both government and the market is a critical 
aspect of the charter school idea, and a method for ensuring that 
charter schools serve both parental and broader societal interests. As 
well, these two forces are intended to combine and create a stronger 
form of accountability then is found in the traditional public school 
system, where schools are less likely to face the possibility of closure 
through either the withdrawal of students or the removal of a contract 
that allows them to operate (although they are increasingly likely to 
face sanctions such as closure or reconstitution if they do not meet 
state-defined performance expectations). The implicit theory of charter 
schools is that these two forms of accountability will complement and 
reinforce each other. 

According to a study of accountability components in charter 
school legislation. Lake and Millot find three general responsibilities 
for charter school authorizers (Lake & Millot, 1998) — (the 
implementation of each of these areas are discussed below). The first 
responsibility is in the charter school application itself, and involves 
the “requirements to become a charter school” (p. 19). Legislation 
varies, but generally includes some of the pieces that must be included 
in an application, such as the school’s mission, the type of staff who 
will be hired, the type of educational program that will be offered. 
Among these requirements are usually the expected outcomes of the 
educational program and some reference to the methods for measuring 
those outcomes. The second responsibility is to monitor or oversee the 
charter school in some way; often, this responsibility rests with the 
authorizer and with one or more other branches of government (i.e. the 
state board of education). Within legislation, specific and/or general 
reports may be required. 

Thirdly, at the core of the charter school theory, authorizers 
must use their authority to choose not to renew a charter for a school 
that has not met the terms of the contract — including expectations 
involving educational performance — or to revoke a charter when the 
operation of a school has clearly strayed from the original intentions. 
(Note 2) In a book on contracting in education, an idea that has some 
strong similarities to chartering. Hill and his colleagues describe a 
contract in this way: 

A contract is a promise to deliver quality education for 
children in return for public funds and a warrant to operate 
a school for some period of time. Some procedure is 
needed to make sure the school lives up to that promise. 

Relying solely on parent choice only holds the school 
responsible for the private benefits of education. (Hill, 

Pierce, & Guthrie, 1997, p. 67) 

In the case of charter schools, advocates have focused their 
rhetoric on renewal as the procedure that will ensure that these schools 
are meeting publicly desirable educational goals. This is consistent 
with Lake and Millot’s argument that, “[e]ffective accountability 
requires an efficient means of terminating schools that fail to achieve 
their contractual requirements, particularly in the area of educational 
outcomes” (Lake & Millot, 1998, p. 20). The combination of 
performance expectations and a “contract” creates a theory of charter 
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school performance accountability that rests on two key assumptions: 

• Authorizers can assess the quality of education offered by 
charter schools, using test scores and, if needed, other methods; 
and, 

• Authorizers will act on their assessments by revoking or not 
renewing charters that do not demonstrate that they are 
providing quality education. 

The “New Accountability” in public education 

While charter schools are generally considered to be outside the 
domain of mainstream educational reform efforts such as standards- 
based reform (cf. Smith & O'Day, 1991), changing ideas about 
educational accountability influence both. Calls for a “new 
accountability,” according to Elmore, Abelmann, and Fuhrman, have 
three central components: “a primary emphasis on measured student 
performance;” “the creation of relatively complex systems of 
standards” used to make comparisons among schools, districts, etc.; 
and “the creation of rewards and penalties and intervention strategies 
to introduce incentives for improvement” (Elmore, Abelmann, & 
Fuhrman, 1996, p. 65). 

Within the traditional system, this new accountability has 
focused on the creation of aligned state standards and assessments that 
are more challenging and rigorous than in the past, and tools for 
recognizing and rewarding schools (and sometimes districts) that are 
rising above expectations and penalizing or offering assistance to 
schools/districts that are failing to meet them. In theory, “focusing on 
student performance should move states away from input 
regulations. . . and toward a model of steering by results” (Elmore et 
al., 1996, p. 65). In the theory underlying charter schools, the primary 
incentives for improving performance are the ability to continue to 
operate as a charter school by having a contract renewed and to be 
successful in the market by attracting students and the public resources 
they bring with them. The reverse of this are the clear sanctions for 
charters if they fail to improve performance — ^the removal of their 
contract, and the loss of students through the market. 

Another facet of this new accountability, at least in some cases, 
is attempts to increase the involvement of parents and communities in 
school reform. This is done largely through the public reporting of 
assessment results. The expectation underlying public reporting is that 
it, “energizes parents and other community members to pressure 
schools for higher performance, particularly when data show 
differences in performance among schools that are roughly comparable 
in the public’s eye” (Elmore et al., 1996, p. 67). Public reporting of a 
range of information, from test scores to attendance and graduation 
rates, has also been a piece of charter school accountability. However, 
in the case of charter schools, the purpose of public reporting has 
primarily been to influence the market; that is, provide information 
that will better enable parents and students to select among charter 
schools and between charter schools and other schooling options. 

While some progress has been made towards the goals of the 
“new accountability” (Education Week, 1999), challenges are still 
abundant; “The reality of educational accountability at the close of the 
century involves contested standards, a problematic distribution of 
authority, weak incentives, variable capacity, and rudimentary 
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technology” (Adams & Kirst, 1999, p. 464). Some of these same 
challenges have emerged for charter sehool authorizers, particularly 
regarding contested standards and variable capacity. 

Performance accountability and charter school 
authorizers 

While the approach of charter school authorizers to issues of 
performance accountability is the focus of this article, it is important to 
embed it within the broader context of the multiple ways in which 
charter school authorizers address accountability. Following a brief 
description of the data used here, I quickly describe how charter school 
applications and oversight by authorizers during the term of a contract 
are used as tools of accountability issues. This is followed by a more 
in-depth exploration of the oversight of educational programs and the 
renewal process. 

Data 



This research draws on a variety of sources of data. However, as 
an exploratory piece, it also raises questions and issues that need 
further consideration, and does not claim to offer a definitive 
discussion of charter school accountability. I draw on two research 
studies that examined charter school authorizers, a study of “New 
Regimes in Educational Governance” conducted for the Consortium 
for Policy Research in Education (CPRE) (Bulkley, 1999a) and the 
national study of charter school accountability conducted by the 
Center on Reinventing Public Education (CRPE) at the University of 
Washington (Hill et at, 2001). Both of these studies included 
interviews with charter school authorizers and charter school 
personnel; the latter involved authorizers and schools in six states 
(Arizona, Michigan, Georgia, Massachusetts, California and 
Colorado), and the former with schools and states in Arizona and 
Michigan. Evidence is also gathered from other recent literature on 
charter schools (i.e. Arsen, Plank, & Sykes, 1999; Center for 
Education Reform, 2000; SRI International, 1997; SRI International, 
2000; Wells & others, 1998; Wohlstetter & Griffin, 1998). 

A report based on the CRPE study was published in 2001 (Hill 
et al., 2001). This study explored a number of different aspects of the 
accountability issue, including market, government and internal 
accountability. That report provides an overview of some of the 
progress and challenges experienced by charter schools as they have 
sought to contend with the many forces placing demands upon them. 

In particular, the report offers a description of the strengthening 

internal accountability found_in_many. schools (or accountability 

among immediate stakeholders in a school, including educators, 
parents, students and community members) and some of the issues 
faced in addressing external accountability, including accountability to 
charter school authorizers. 

Applications 

The first formal stage for any prospective charter school 
operator is to submit an application to a public organization allowed to 
authorize charter schools. The expectation of many policy makers and 
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advocates of charter school laws was that these contracts would have 
very explicit performance objectives (Bulkley, 1999b). However, 
research suggests this if often not the case. For example, Hannaway’s 
work on educational performance contracting suggests that specificity 
for performance in educational contracts is often low (Hannaway, 
1999), and one study in California found that goals in contracts ranged 
from concrete and quantitative to informal and process-oriented (SRI 
International, 1997; see also Wells & others, 1998). In Colorado, 
charter school applications must explicate student performance 
standards, measurable objectives for student growth, and assessment 
and reporting procedures. But, in practice, some plans are very specific 
while others are “less susceptible to easy measurement” (Clayton 
Foundation, 1999, p. 51). 

When charter school laws were first adopted, authorizers or 
potential authorizers with little or no experience in granting 
performance contracts were placed in a position of wanting or needing 
to evaluate and approve applications almost immediately. In addition, 
some authorizers were under extreme political pressure to allow some 
schools to get up and operating quickly. In this unclear and sometimes 
harried environment, applicants often went through a fairly minimal 
process (c. f Bulkley, 1999a). (Note 3) The national situation may be 
improving, however. For example, every chartering agency in SRJ’s 
national survey reported that some or all of its schools had measurable 
goals in the area of student achievement (SRI International, 2000). 

Since those early years, authorizers with more experience have 
developed clearer guidelines for applicants and, in some cases, clearer 
guidelines for the evaluation of applications. Many of these authorizers 
now require information that, they hope, will help to determine if 
applicants will be equipped to handle the administrative, financial and 
educational aspects of operating a charter school. (Note 4) 

For some authorizers working with a large number of schools, 
the process of granting a charter is seen as a way to influence school 
quality both by selecting the most promising applications and by 
having an application process that builds capacity. Thus, these 
authorizers believe, the process itself can improve the quality of the 
schools that open by forcing applicants to address some issues 
involved in operating a charter school that they might not have 
considered previously. According to a staff person at Central Michigan 
University, in the application process, “we’re going to take you 
through a structured process, we’re going to help you anticipate 
operational difficulties, we’re going to help you prepare your 
organization so that the first day that you open the doors, you’re going 
[to] be prepared to educate kids.” 

Implicit in this focus on applications as an accountability 
mechanism is a belief, at least among staff at some of the larger 
authorizers, that if they make the process rigorous enough at the 
beginning, then they won’t need to “worry” as much about the school 
in practice. While the theory of charter school accountability has 
generally focused on some interpretation of student outcomes, staff 
working for some authorizers expressed a belief that charter schools 
can be more accountable because of the initial application process. The 
more rigorous the process, they argue, the more accountable the 
school. 

Oversight 
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Once an authorizer grants a charter and a new school opens or a 
pre-existing school begins to operate with a new governance structure, 
the authorizer is responsible for overseeing the school in a number of 
different areas. The authorizer must determine if a school is compliant 
with any applicable laws and regulations, as well as any specific 
provisions in the charter document. As a part of compliance, the 
authorizer needs to examine the finances of a school to check if they 
comply with spending and bookkeeping requirements and determine if 
schools are “functioning organizations.” The following section 
examines their oversight of the educational programs of charter 
schools during the contract period. 

Authorizers use a variety of tools for oversight, including 
required reports, site visits, parental complaints and surveys, outcome 
data, regular meetings and informal contact. There is considerable 
variation between authorizers as to the types of tools used and the 
frequency with which they are used, ranging from those who have very 
minimal contact with schools (generally limited to written reporting 
unless major problems arise) to others who supplement reporting with 
regular contact through visits, meetings and phone calls. 

The most basic level of oversight for charter school authorizers 
is ensuring that schools meet legal requirements regarding compliance 
with state, federal, and local laws and regulations, with acceptable 
accounting practices, and with reporting requirements. While the 
theory underlying charter schools in most states revolves heavily 
around student achievement and school performance, a number of 
studies suggest that authorizers often focus their oversight on the 
familiar, such as compliance and financial stability, rather than on 
performance (Bulkley, 1999a; Gam, 1998; Henig, Moser, Holyoke, & 
Lacireno-Paquet, 1999; Hill et al., 2001; SRI International, 1997). In 
one California study, “school district officials note that, given all the 
ambiguity around student outcomes and what measures are valid, they 
are holding charter schools accountable more on fiscal, rather than 
academic, measures” (Wells & others, 1998, p. 19). 

Regardless of whether or not compliance requirements directly 
impinge on the abilities of a school to operate the educational program 
it desires, there is clearly an opportunity cost for the schools related to 
compliance; the more time spent on regulatory requirements, the less 
time and money available for other purposes (Arsen et al., 1999). 

Alongside general oversight, there is a subset of charter schools 
that require additional attention from their authorizers. These “non- 
functioning organizations” are experiencing major problems such as a 
substantial loss of students and/or staff, considerable infighting among 
staff, parents, board members or others in the school community, 
severe student discipline issues or major financial problems. 
Authorizers can identify schools that are completely, or in some aspect 
of their operations, non-functional through a number of sources. These 
include parental complaints, site visits, financial audits or reporting, 
and media “exposes.” Often, multiple sources inform an authorizer 
that a school is having serious struggles. 

In some cases, authorizers do nothing but simply monitor the 
situation unless or until it becomes severe. Other times, however, they 
become more engaged with the troubled school, often “behind-the- 
scenes.” For example, authorizers can work directly with school 
boards and school leaders to attempt to resolve problems. In Michigan, 
some universities have taken an active role — sometimes apparent and 
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sometimes behind the scenes — in schools that have been struggling; 
for example, authorizing staff have helped to identify governance 
problems in schools and promoted the replacement of board members 
and school leaders through advice to boards and leaders. Similarly, in 
Colorado, the school leader in one school visited for the CRPE study 
left in part because of pressures from the sponsoring district. 

Finally, regular monitoring combined with detailed feedback to 
schools can be useful for authorizers and struggling charter schools. In 
Massachusetts, in-depth school site visits are one opportunity to assess 
how well a school is functioning. The two schools in the CRPE study 
authorized by the Massachusetts Board of Education both had 
difficulties early on, one where problems where largely governance- 
oriented, and another where they primarily involved the school’s 
educational program. In each case, the site visit reports provided a 
form of technical assistance to the schools. By seeing themselves 
through the eyes of outside experts, school personnel were more able 
to identify and address their internal problems. 

Examining the educational program and performance 

In the theory underlying charter schools in most states, the 
schools are expected to offer an educational program that leads to 
improved student achievement (Lake & Millot, 1998). Authorizers can 
look at the outcomes of a school’s education program only at the time 
of renewal or formal review, or they can monitor this program 
throughout the charter contract period as well. 

Monitoring during the Contract. Early on in their work with 
charter schools, authorizers often focused their energy on ensuring that 
schools were functional organizations and were in compliance with 
fiscal and regulatory requirements. As authorizers have gained 
knowledge and experience with compliance issues, the time required 
for addressing these issues has been minimized and become more 
routinized. This has allowed them to turn increased attention to the 
educational programs offered by schools. 

Authorizers use a number of different tools in monitoring 
academic achievement and a school’s educational program. The most 
common, and certainly the most publicly visible, is student test scores. 
Some authorizers merely collect test score data, but do little with it 
during the charter contract period. In other cases, authorizers analyze 
the scores and provide information to the public and/or school 
personnel about these outcomes. A recent study by RPP International 
found that 85% of charter schools reported test scores to their 
authorizer (RPP International, 2000). However, it is important to note 
that the fact that student achievement is monitored, or tracked, does 
not necessarily mean that this data is utilized in decision-making and 
other actions by the authorizer. For example, in a study of California 
charter schools, 85% schools said they reported student achievement 
data to their sponsor, but only 4% said that the sponsor “had ever 
requested specific actions or imposed sanctions in response . . ..” (SRI 
International, 1997, Part II, p. 16, emphasis in original). This potential 
lack of use of achievement data is tied closely to a lack of clear 
standards for schools; as Wohlstetter and Griffin found, “sponsoring 
agencies, in general, required assessment information on performance 
from charter schools. . . but often failed to specify any clear 
performance standards or consequences” (Wohlstetter & Griffin, 1998, 
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p. 15). 

Test scores are not the only information used by authorizers to 
assess a school’s educational program, as the following two examples 
demonstrate. In Massachusetts, where charter schools negotiate an 
accountability contract with their authorizer at the beginning of their 
charter, schools submit an annual report that includes, among other 
things, a discussion of their progress towards the goals of their 
accountability contract. Authorizers in several other states also require 
aimual progress reports; these vary from one-page commentaries to 
more elaborate formal reports that include specific examples and 
evidence demonstrating educational progress. In Michigan, one 
university has recently been pushing charter schools to develop goals 
that are “clear, concise and measurable,” in response to concerns that 
goals in charter contracts have at time been imprecise and progress on 
them difficult to assess. However, it is unclear what, if any, formal 
actions are taken if a school’s educational program is seen as 
inadequate by authorizer staff. 

Another way of assessing a school’s educational offerings and 
aiding in improvement is through the use of outside organizations. For 
example, there a number of organizations that either accredit schools 
or are planning to offer accreditation, including several state charter 
school organizations. Two school districts in the CRPE study have 
required that the schools they authorize become accredited by their 
respective state charter organizations when this becomes possible. 

In general, during the period when a charter contract is in effect, 
authorizers may be more focused on the day-to-day issues of 
compliance and dealing with non-functioning organizations than they 
were on broader issues of accountability for the education program 
schools offer. Even within the domain of “educational accountability,” 
authorizers cite actions such as ensuring that teachers are certified and 
the school’s curriculum is properly aligned with state standards — 
while these certainly may influence the academic program offered by 
schools, they are not the kinds of “new accountability” tools that 
charter advocates have emphasized. Nationally, such monitoring of 
performance has seldom led to charter revocation (SRI International, 
2000 ). 

Renewal. The renewal process for charter schools varies 
considerably across states and across authorizers. The process and 
criteria for renewal can be fairly clear and defined, or undefined to the 
point of great confusion for school personnel. The renewal process has 
the potential (not necessarily attained) to be a serious undertaking that 
holds the genuine possibility of a school not receiving a new charter, 
thus providing schools with a strong external incentive. 

The Massachusetts Department of Education is probably the 
authorizer whose renewal process is most frequently held up as a 
“model.” In Massachusetts, a number of different data sources are used 
in the renewal process. The most elaborate piece involves an 
evaluation/renewal inspection by an outside team hired by the 
authorizer. These teams, which consist of experienced educators and 
others, conduct 3-4 day visits at each school up for renewal and write a 
detailed report on their findings for the authorizer. A number of 
schools have come up for renewal, and all have received new charters 
along with suggestions for improving their programs. 

Among other large authorizers, few have developed as clear or 
rigorous a procedure as has the Massachusetts DOE. For example, in 
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the spring of 1999, one Michigan university’s office that works with 
charter schools recommended that the university board renew all the 
schools whose contracts were due to expire. The materials provided to 
the university board included test score data on the MEAP (the 
Michigan state test) and another national standardized test, alongside 
other information on school goals. The test scores were mixed, with 
some schools and grades showing marked improvement and some 
showing little or negative change; how good is “good enough” was not 
spelled out either formally or informally to these schools or the 
university board. Other goals tended to emphasize participation rather 
than performance, such as attendance in foreign language or computer 
classes. At another Michigan university that has authorized a 
significant number of schools, staff said that they expected all their 
schools to be reauthorized before the renewal process had even begun. 

Small authorizers (those that have granted only a few charter 
contracts) vary considerably in their approaches to renewal and are 
unlikely to be as clear or thorough as larger authorizers that have more 
resources and capacity at their disposal. Some have few clear 
procedures and criteria for renewal. Among the case study schools, 
those whose charters were granted by small authorizers were generally 
not very concerned about renewal. For example, personnel at the 
Georgia Department of Education have emphasized to the two CRPE 
schools the need for schools to have specific performance outcome 
goals in their charters, yet there was little focus on performance as an 
aspect of the renewal process at either school. 

Some of the small authorizers have or intend to utilize external 
information sources in their renewal decisions, including accreditation 
(discussed above). For example, one district in Colorado hired an 
independent evaluator to evaluate a school in the year it was being 
considered for renewal. One California district used an external review 
of a charter school that was done for other district purposes in its 
renewal decision alongside an external study completed specifically 
for the renewal process. While authorizers are using a variety of tools 
to evaluate a school when its charter is up for renewal, the authorizers 
in the CRPE study generally expected to renew the charters they had 
granted when the time came. 

Overall, for many authorizers, accountability to government has 
primarily focused on issues of financial and legal compliance, with 
some monitoring of educational programs (primarily through test 
scores). However, it is unclear to what extent government authorizers 
are using educational monitoring or evaluation to make serious 
decisions about the operation of individual schools. This challenge to 
performance accountability is recognized by both advocates and critics 
of charter schools; as one study noted, in practice, "accountability 
typically means a half-baked version of the top-down regulation-and- 
compliance system that the state or community applies to its 
conventional public schools" (Finn et al., 2000, p. 135). 

The Accountability Bind 

Charter school authorizers vary in important ways, such as their 
emphasis on compliance vis a vis oversight of educational programs, 
their beliefs about the proper role of an authorizer in the day-to-day 
operations of schools, and their faith in test scores as an appropriate 
measure of quality. These differences can be explained as a 
combination of differences in the will and capacity of authorizing 
agencies. For example, the charter schools office in the Massachusetts 
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Department of Education has operated in a political climate where 
there is a considerable interest in close monitoring of charter schools, 
and the office has responded in kind. This contrasts considerably with 
the two state boards in Arizona, where there is little political will for 
strong oversight; that political will has been translated into the 
selection of staff who are supporters of a more “laissez-faire” approach 
to authorizing, and into minimal funding for charter staff which 
effectively limits the oversight potential regardless of the inclinations 
of the staff members. 

As the study by SRI International notes, while the vast majority 
of charter schools have yet to come up for renewal, those that have are 
generally receiving new contracts (SRI International, 2000). This is 
consistent with the information provided by the school and authorizing 
agency staff interviewed for the CRPE study. Interviews with 
authorizers and a review of the literature on charter schools suggest 
that they all share a number of significant challenges to closing 
schools, despite considerable and very important variation across 
authorizers in their approaches to the renewal issue. These challenges 
are tied to the some basic assumptions underlying the idea of charter 
school accountability. Four challenges are: 

1 . Educational performance is not simple to define or measure, nor 
is how good is “good enough” in educational quality. 

2. Other aspects of a school’s program, often more difficult to 
measure than test scores, are also important to families and 
authorizers. In this context, authorizers sometimes turn to 
“proxies” to assess school quality. 

3. Teachers, parents and students become very invested in 
particular schools and destroying a community may be more 
difficult for authorizers than serving a diffuse public interest. 

4. Finally, charter schools have become a highly politicized issue 
on both sides, and some authorizers are concerned about their 
decisions reflecting poorly on charter schools as a reform idea. 

I call this constellation of challenges the “accountability bind,” 
as authorizers are stuck between wanting to enforce accountability 
through renewal, but finding doing so fraught with difficulty. I am not 
suggesting that no authorizers are taking performance accountability 
seriously — some clearly are - but that acting as the originators of 
charter schools intended can be very difficult for authorizers. 

Challenge 1: Defining educational performance with test scores 

One of the greatest challenges for states attempting to develop 
accountability systems as part of standards-based reform is to create 
systems that are considered by those within education and the public to 
be fair and defensible (Elmore et al., 1996). Thus, in order to attach 
consequences to performance measures, they must be viewed as valid 
and reliable measures that appropriately distinguish between schools 
where a desirable level of learning is t^ing place and others where 
learning is inadequate. For charter school authorizers, these demands 
of fairness and defensibility are even greater than for state assessment 
systems, because the sanction of revoking or not renewing a contract is 
so high. This challenge sits alongside the challenge of using 
standardized tests to assess the performance of individual charter 
schools, which often aim to offer non-traditional educational 
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programs. While authorizers are clearly monitoring the test seores of 
students in charter schools, it is extremely important to distinguish 
between gathering information about school quality and using that 
information for improvement and accountability purposes. Some 
authorizers have worked with schools to develop expectations that are 
more aligned to the school’s stated purpose than an off-the-shelf 
standardized assessments, but this makes comparisons with other 
schools - one way of determining if a school is successful - all the 
more challenging. As well, charter schools often must adapt their 
programs to serve the needs of students who enroll at the school, who 
may or may not match the students expected by the founders; this can 
lead to a situation where meeting specific contract goals may be an 
unreasonable expectation. As the Hill et al study notes, “Finding ways 
to measure not only student achievement on standardized tests but the 
value-added qualities of charter schools has proven to be a 
challenge” (Hill et al., 2001, p. vi). 

Challenge 2: Assessing quality beyond test scores 

Authorizers' uncertainty about closing schools is also tied, at 
least in some cases, to staff members who feel that the essence of a 
school can never be captured by quantifiable measures, and who 
instead rely on a sense of “feel” about the quality of the school. As one 
staff person said, “[you need to] look into the eyes of the kids and the 
teachers and if you see the magic. . . you know you’ve got something 
good happening.” While this attention to "feel" may well capture 
aspects of the educational experience offered by a school that would be 
missed by test scores and other such measures, it is inconsistent with 
the idea that charter schools should continue to operate based on their 
ability to demonstrate academic performance. 

The challenges of determining “success” have often led 
authorizers to “proxies” that, they believe, provide indicators about the 
performance of individual schools. Interestingly, proxies generally 
seem to be used to justify an authorizer’s assessment of “success” in a 
school, and rarely to argue that a school is not doing well. The most 
obvious proxy authorizers use is parental choice and satisfaction. 

While markets and government authority are intended to work 
separately in creating accountability for charter schools, authorizers — 
when asked to explain success — frequently point to the market. For 
example, the existence of waiting lists for slots at a charter school is 
often given as proof or evidence of the school’s worth. The challenge, 
of course, is that if market accountability is created (obviously) via the 
market, and government is turning to the market to demonstrate 
success or performance, than the ideal of joint accountability for 
performance between markets and government is compromised. 

Authorizers also point to the “mark of approval” given by 
external organizations as evidence of success or an increased 
likelihood of a suceessful school. One such mark is accreditation, both 
from organizations that pre-date charter schools (i.e. accreditation 
from the North Central Association Commission on Accreditation and 
School Improvement ) and those that have sprung up specifically to 
work with these schools (mainly state charter school associations). The 
approval given by these organizations is a sign of quality in the eyes of 
many authorizers, although the lack of accreditation is not generally 
seen as problematic. (Note 5) 

Finally, educational management organizations (EMOs), 
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sometimes called service providers, have become increasingly 
common in a number of states (in Michigan, over half of all charter 
schools now use a service provider). These organizations can offer a 
variety of services, from simply keeping track of a school’s finances to 
running the entire school, including providing the facility, hiring the 
staff, and selecting the educational program. Comments by authorizers 
suggest that, when one school operated by an EMO is seen as 
successful, than other schools run by the same company are more 
likely to be seen as successful. As with the proxy of accreditation, this 
reliance on EMOs may or may not be appropriate, but it is still an 
indirect mechanism for determining success. These examples suggest 
that proxies for quality often involve organizations or individuals that 
are largely outside the government domain, and do not themselves 
have a direct obligation to serve public purposes. 

Challenge 3: The preferences of charter school communities vs. 
the “public interest” 

WTiile authorizers have difficulty with determining what is and 
is not a successful charter school, they have even more difficulty 
deciding that a school is unsuccessful enough to justify as high a 
sanction as closure. Authorizers serve a diffuse public that has general 
needs for high quality education, but their day-to-day reality centers 
around working closely with charter school operators and communities 
of teachers, parents and students who are heavily invested in the 
continuing existence of a particular school. As one staff member with 
an authorizer said: 

The theory sounds great about shutting schools down. The 
practice is much more difficult. And whether you’re 
revoking a contract or not renewing a contract, they’re 
both gonna be tremendously difficult. . . the reality is 
you’ve got teachers, you’ve got parents, you’ve got a 
community that’s now used to the school, depending on 
the school, loving the school and either way you’re gonna 
have difficulties. 

This attitude towards renewal appears to be common among 
authorizers, despite their recognition of the political rhetoric arguing 
that charter schools should not be renewed if they can’t prove they are 
improving achievement. The turmoil created for a specific group of 
educators, parents and children that results from not renewing a charter 
may outweigh the difficult to define the ideal of performance 
accountability for authorizers. Their relationships with charter 
communities bring them into regular contact with parents who are 
convinced of the value of individual schools and schools that have 
long waiting lists (with or without “proof’ of academic quality), and 
this positive response of the market can have a profound impact on the 
thinking of authorizers, despite the ideal separating accountability to 
the market from that to government. 

Challenge 4: Politics and the charter school movement 

Finally, a number of authorizers are themselves politically 
invested in the success of the charter school “movement.” The 
authorizers I have studied, particularly those who are involved with a 
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large number of schools, are generally staffed by people sympathetic 
to charter schools. Even if they personally support the ideals of 
performance accountability, they may be hesitant to close schools 
because of fear that this will be seen as a failure of the general charter 
school ideal. 

Finding the middle ground 

Overall, the forces working against the closure of charter 
schools based on student performance seem likely to result in schools 
being renewed at a very high rate. What seems to be needed is a less 
all-or-nothing definition of accountability — ^more of an ongoing 
relationship than a single decision point. There are a number of ways 
authorizers are currently working to establish more productive and 
educationally substantive accountability relationships with ch 2 irter 
schools through “middle grounds” that allow them to take some action 
short of actually revoking or not-renewing a charter contract. These 
middle grounds include: focusing on applications as a form of “input” 
accountability, direct or indirect intervention in schools, and capacity 
building/teclmical support. 

As described earlier, authorizers (especially large authorizers) 
have increasingly emphasized the importance of the application 
process as a quality control mechanism. When asked about how 
schools are held accountable, several staff members working with 
authorizing entities responded in part by discussing the rigors of the 
application process. In light of the difficulty of closing schools, it 
makes sense that authorizers pour considerable time and energy into 
doing what is in their power to see that new charter schools have the 
greatest potential to be successful. 

Authorizers can also intervene once schools have begun to 
operate, especially when problems are present, through direct or 
indirect methods. Direct intervention involves requiring that schools 
do something. These interventions often involve compliance issues 
and schools that are struggling to be functional; for example, a school 
with financial problems might be told to undergo an external audit. A 
study by SRI International examined “corrective actions” taken by 
authorizers, including not only revocation and non-renewal but also 
probationary measures; they found that such corrective actions were 
most often related to fiscal or management issues (SRI International, 
2000). Direct interventions linked to the educational program appear 
less common, but do occur. For example, in Arizona, the State Board 
of Education required all schools it had authorized that had average 
test scores below the 35th percentile to provide information explaining 
these scores. (Note 6) While direct interventions happen, indirect 
interventions seem far more common. These often take the form of 
discussions about problems between school and authorizer staff, 
including problems around issues such as staffing, curriculum, test 
scores, and goals. 

Finally, authorizers seek to find a middle ground by providing 
schools with technical assistance or other capacity-building measures 
with the hopes that the increased capacity will result in higher 
achievement and better functioning organizations. Technical assistance 
can include professional development or, more often, suggestions of 
where to turn for professional development. Another form of capacity- 
building is to send external reviewers to schools, with the expectation 
that they will provide summative evaluations and formative 
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evaluations that offer suggestions and insights that school personnel 
might find useful. This form of review has been used since the 
beginning in Massachusetts, and has also appeared in a variety of other 
states. For example, one California charter school authorized by a 
district that is very focused on test scores and student achievement 
experienced declining scores. The district responded by providing an 
external review designed to offer suggestions for improvement. 

Finally, authorizers can work to build capacity among parents for 
making good choices (i.e. through providing information), attempting 
to improve quality through improving parental decisions in the 
educational market. 

One of the challenge of technical support designed to help 
schools improve themselves, however, is similar to that faced by states 
and districts trying to reform "under-performing schools" — it is not 
always clear to either authorizers or schools what exactly is needed to 
improve performance. This challenge of an “unclear 
technology” (Tyack, 1993) is obviously not unique to the charter 
context, but it may be exacerbated by the separation between charter 
schools and other entities that may be in a position to support change 
and provide expertise (i.e. school districts). As Elmore and his 
colleagues comment, “performance-based accountability systems 
depend heavily for their success on whether school administrators, 
teachers, parents, and students know what to do to improve 
performance” (Elmore et al., 1996, p. 92, emphasis in original). 

Attempts by authorizers to find a middle ground, some set of 
actions that allows them to influence charter schools without taking 
the undesirable step of closing them, has led to a variety of creative 
approaches. However, there is a serious challenge for the theory of 
charter school accountability in an increased focus on middle grounds 
and a lesser focus on contract renewal. 

The way in which charter advocates have presented the idea of a 
performance contract is, fundamentally, as an “all or nothing” 
proposition. In arguing for a broad-based system of contracting. Hill 
and his colleagues make the case that, “Contractors who failed to 
provide instruction as promised, or whose students’ outcomes were 
low and not improving as anticipated, could be fired or given an 
ultimatum to improve or be replaced” (Hill et al., 1997, p. 70). In 
addition, they argue that, “accountability only means something if 
there are consequences when children do not learn” (Hill et al., 1997, 
p. 68). The rationale behind the additional autonomy provided to 
charter schools is that there will be real and significant consequences if 
performance is not demonstrated. 

If, as appears to be the case, renewal is not always (or even 
often) serving its intended function as a mechanism for performance 
accountability, the autonomy /accountability bargain on which charter 
schools are based is out of balance. Given the problems inherent in an 
all-or-nothing renewal decision, the goal is to make sure that 
accountability for educational performance is maintained throughout 
the contract by building in a range of incentives, supports, and 
consequences less drastic than non-renewal. The challenge is to do this 
without the authorizer becoming so involved in the day-to-day 
operations of charter schools that the autonomy that makes them 
distinctive is threatened. 
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Conclusion 

Charter school authorizers, in the years since states began 
passing charter school laws, have had to develop a set of procedures 
and standards to work with public schools governed in a very different 
way than the hierarchical, bureaucratic method traditionally followed. 
While their focus has rhetorically been on school performance, they 
have come up against many of the challenges that the public education 
system has faced in determining performance and meting out rewards 
(continued operation in the case of charter schools) and punishments 
(revocation or non-renewal). 

However, internal accountability mechanisms in charter schools 
may be compensating for the lacking external accountability 
envisioned by some charter advocates; internal accountability 
involves, “a set of productive and mutually responsible relationships 
among teachers, administrators, and parents, united on behalf of 
effective instruction for children” (Hill et al., 2001, p. iv). The charter 
school authorizer-charter school relationship has shown itself to be 
different in most cases than the traditional school-district relationship 
(with the most common exceptions being districts authorizing public 
school conversion charters). However, it has often not met the ideals 
of those charter advocates who envisioned a rigorous authorizer 
accountability system that only continued to support those schools that 
had unequivocally improved student achievement. 

So, where does this leave us? These findings are preliminary, 
and other states with more explicit laws may have authorizers relying 
more heavily on renewal and revocation as accountability 
mechanisms. However, the incentive structure that is operating for 
authorizers described here seems likely to apply to authorizers more 
generally. In this environment, one could simply argue that the 
fondamental theory of charter schools is flawed, and thus this reform 
effort should be placed in the "loss column." 

On the other hand, one can turn to other alternatives to support 
charter school accountability, such as internal accountability and the 
marketplace and one can focus on middle grounds, including those 
discussed here and others that will be created over time as authorizers 
grapple with the challenges of using renewal and revocation or 
accountability. One possibility, which has its own difficulties, would 
be to separate authorizers from those who make decisions about 
renewal. In theory, this would remove the political disincentives of 
non-renewal and the personal relationships that can convolute renewal 
decisions. However, a risk in this approach is that authorizers who 
have developed relationships with schools are likely the entity in the 
best position to assess the quality of an individual school. 

Charter schools are simply one form of contracting in public 
education. Other forms, such as when a school district contracts 
directly with a private provider, may encounter similar challenges, i.e. 
the uncertainty of assessing quality and the reticence to invoke high 
stakes. However, there are some important differences. The most 
important such difference is that the stakes in the case of charter 
schools are so incredibly high. When a district contracts with a private 
provider, ending the contract generally does not mean ending the 
existence of the school. In this environment, the theory of high stakes 
and the actuality of high stakes for private providers may be more 
closely aligned. (Note 7) 

In this article, I have questioned one of the fundamental 
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elements of the charter school theory — ^that accountability to 
govermnent can be achieved through "all or nothing" contractual 
arrangements. The accountability bind that this creates for authorizers 
seems unlikely to change. Thus, charter school advocates may be well 
advised to reevaluate and adjust their theory — and possibly their 
rhetoric and legislation — in light of the questionable value of such a 
heavy reliance on revocation and renewal for ensuring performance 
accountability. The real risk is that there will be no change in the 
status quo, and the ideal of charter schools as jointly accountable to the 
market and government for educational performance and quality will 
simply be lost, with the market dominating the accountability 
equation. 
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Notes 

1 . Arizona and the District of Columbia allow charters of up to 15 
years. 

2. All states allow charters to be revoked or not renewed for 
reasons related to educational performance. However, allowing 
performanee to be a criterion and requiring it are different 
things. Several states (none of which were included in the CRPE 
study), including Texas, Louisiana and New Hampshire, require 
in their legislation that schools must demonstrate acceptable 
levels of performance or improvement, based at least in part on 
test scores (Lake and Millot, 1998). It is unclear what effect this 
will have on renewal in those states. 

3. One clear exception to this was applications submitted to the 
Massachusetts Board of Education, which had a more elaborate 
application process from the beginning. 

4. The application process can include intensive interviews with 
prospective authorizers, site visits to potential facilities, and 
reviews of the intended curriculum and personnel or type of 
personnel to be hired. 

5. This appears to be changing, at least in some cases, as more 
authorizers are requesting that their schools go through an 
accrediting process. 

6. For one school that served a largely at-risk student population, 
the 35% cut-off point was seen as arbitrary and inappropriate as 
a tool forjudging their success with their particular student 
body. 
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7. While charter school advocates have tended to be skeptical of 
public school conversions, such schools may in fact be more 
accountable in certain ways than "new start" schools because 
there is the real possibility that the charter could be removed 
since the school itself would continue to operate. 
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Abstract 

The following narratives examine three teachers 
over a course of ten years as they first entered teaching 
and began to collaborate with other teachers on 
curriculum. Specifically, the study examines how the 
teachers 1) developed as collaborators and 2) perceived 
elements of support from both within and outside the 
classroom for their collaborative efforts. The article 
argues that the successful collaborative efforts helped 
deepen their sense of agency and initiative within their 
teaching and, to a lesser degree, stimulated reform and 
change within their schools. In turn and to varying 
degrees, the process of collaboration supported their 
personal renewal in their work. The article suggests that 
structural support for these teachers that connected to their 
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emerging personal practical knowledge was crucial for 
their development as teacher collaborators. The article 
concludes by suggesting how schools may be restructured 
to start to become sites of authentic leadership that build 
on the talents, meaning, voice, and knowledge of teachers. 



Teachers are often viewed as sentries to change, working alone 
in their classroom, repeating daily routines, delivering well-worn 
lessons. Outside their classrooms, traditional school cultures and 
structures prompt these pursuits by reinforcing the present, the 
conservative, and the individual (Lortie, 1975; Sarason, 1982). Given 
these constraints, an implicit debate has emerged in the literature on 
school change about the anonymous teacher as an instructional leader 
at her or his school. One view supports the notion that teachers can 
develop and use capacity for initiative and change at the local level 
(Cuban, 1998). In this view, teachers may work critically, reflectively, 
and ethically (Greene, 1989) in ways that support a sense of authorship 
in their teaching life (Greene, 1989; Sawyer, 2001, Vinz, 1996) and 
change in their settings (Wasley, 1 992). 

Another position, however, suggests that the culture of schools 
often prevents teachers from following personal initiatives to work 
together for personal renewal and school reform (Sarason, 1982, 

1990). A comparison between the work of university faculty and that 
of K-12 faculty helps clarify this discussion. University faculty are 
rewarded when they engage in inquiry that may both support new 
knowledge and facilitate change (a synergy between research and 
service). K-12 teachers, however, are rewarded by implementing 
curriculum that supports various state mandates, mandates which often 
do not align with the personal meaning that teachers find in their work. 

The following study examines how three teachers (one middle 
school and one high school mathematics teacher and two high school 
English teachers) deepened their development of agency and initiative 
to work towards personal self-renewal and school change by 
collaborating with their peers. These teachers were each anonymous in 
the sense that they were not department chairs or members of any 
organized teacher groups. They also all worked in schools that may be 
characterized as having non-collaborative cultures. 

The following questions guided this study: What did some of the 
teachers’ collaborative structures look like over time? How did the 
teachers perceive contexts of support from both within and outside the 
classroom for their collaborative efforts? And, how did this support 
change at different points as these teachers' careers unfolded? The 
larger issue of how teachers through their own initiative can work 
together for personal renewal and school reform frames these 
questions. 

Supporting Teachers Who Collaborate 

A balance of conditions and elements undergird more successful 
approaches to teacher collaboration. These elements include school 
cultures, department sub-cultures, the development of meaningful 
content in context, and specific resources, such as time. Little is 
known, however, about how teachers who emerge as leaders find and 
structure support for their activities. In addition, little is known about 
how these elements may change or unfold at different points in their 
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careers. 

School Cultures 

School settings with norms of collaboration greatly support 
teachers who collaborate (Lieberman, 1995; Little, 1982, 1993a; 
McLaughlin, 1993). Such norms go beyond social interaction to 
indicate innovation and learning "in which teachers are enthusiastic 
about their work and the focus is on devising strategies that enable all 
students to prosper" (McLaughlin, 1993, p. 94). Cultures of 
collaboration facilitate "a sense of mutual security and encourag[e] 
interpersonal and interprofessional openness" (Nias, 1989, p. 2). A 
number of characteristics distinguish collegial schools. Elements 
include teachers' frequent and concrete talk about teaching practice; 
frequent and honest observations of teaching; the collaborative design, 
research, and evaluation of teaching materials; and peer teaching and 
coaching of teaching practice (Little, 1982, p. 331). Key to the 
formation of norms of collegiality is making "development of effective 
instructional practices for all students the top priority" (McLaughlin, 
1993, p. 96). Ideally, the setting supports a community of "collective 
responsibility — of mutual support and mutual obligation — ^for practice 
and for student outcome" (McLaughlin, 1993, p. 97). 

The character of the school's collegial environment matters as it 
fosters mutual problem solving and planning (Hargreaves, 1996). 
McLaughlin (1993), states; 

teachers within the same school or even within the same 
department developed different responses to similar 
students depending on the character of their collegial 
environment. Which response a teacher chose was a 
product of his or her conception of task as framed and 
supported by a particular school or department community 
(p. 89). 

This process involves a complex match between school and 
teacher goals and school support for teachers' conceptions of their 
meaningful practice (McLaughlin, 1993). 

Department Subcultures 

While these influences can take place on a school-wide level, 
considerable variation in levels of support and teachers' responses to 
students can take place on the departmental level. Leadership on a 
departmental level helps determine whether and how teachers 
collaborate (Hargreaves, 1996; McLaughlin, 1993). Indeed, subject- 
matter departments can create subcultures with distinct approaches to 
curriculum and pedagogy within the same school (Grossman & 
Stodolsky, 1994; McLaughlin & Talbert, 1993; Siskin, 1994). Given 
the subject-matter organization of secondary schools, departments can 
represent an important context for teacher interaction, the "most 
prominent domain of potential interdependence among 
teachers" (Little, 1993b, p. 149). The department can be the 
"professional community of greatest significance to teachers' norms of 
practice, conceptions of task, and attitudes toward teaching and 
students" (Siskin, 1990, as cited in McLaughlin, 1993, p. 92). The 
character of the departments — its norms of collegiality — ^plays a key 
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role in the way teachers construct their practice and relate to students. 
The clarity of vision of the department can also help focus the 
collective and individual curriculum response to students (Ball, 1987; 
Ball & Bowe, 1992). Given the central position of departments to 
teachers' interactions, departments represent a potential to limit forms 
of interdisciplinary and cross-departmental forms of interaction. Little 
(1993b), for example, found that limited cross-departmental 
collaboration existed within survival-oriented departments whose 
teachers worked together only to secure resources for themselves. 

Meaningful Content in Context 

A further support for teacher collaboration stems from how well 
teachers perceive that the collaborative work actually has meaning for 
them in their work with students. While most educators support the 
process of collaboration for teachers, some question whether teacher 
collaboration is authentic or contrived (Hargreaves and Dawe, 1 990), 
explicitly professional or implicitly communal (Huberman, l993), and 
pedagogically sound or undermining of more spontaneous, 
idiosyncratic, and context-specific teaching methods. Huberman 
(1993), for example, suggests that teaching is highly context specific 
and personal: "To plan collaboratively...is to reduce the degree of 
freedom required for the multitude of context-sensitive, continuously 
evolving, interactive responses that many teachers call on" (p. 1 9). 

This concern assumes that teachers carmot either explicitly articulate 
or gain the perspective necessary to reveal their classroom dynamic, 
instead often engaging in a unique form of "communion." Supported 
by how similar teachers' teaching philosophies and approaches are and 
by a lack of explicit teacher reflection, this form of collaboration might 
serve to reinforce existing forms of teaching without promoting self- 
reflection or problem-solving behavior. 

Related to the view of teaching as context-specific and 
idiosyncratic is the issue of the actual substance of the collaboration. 
Huberman (1993), in referring to the teacher as artisan, suggests that 
teachers who collaborate take a more "tool-centered" rather than 
substantive approach. A study by Zahorik (1987) of 52 teachers in six 
schools supports this view. Seventy percent of the time he found a 
student focus to the teachers' collaboration: materials, discipline, 
activities, and individualization, reflecting, in his view, an emphasis on 
student behavior. Collaborating teachers were less willing to discuss 
topics with more of a substantive teaching focus: evaluation, methods, 
objectives, reinforcing, lecturing, questioning, and room organization. 
Reasons that might encourage teachers to refrain from exchanging 
information about teaching strategies include the maintenance of 
professional respect for the core work of peers (Bishop, 1977), the 
tolerance of individual preferences and styles (Little, 1990), and the 
avoidance of arrogance (Huberman, 1993). 

Many of these criticisms underlie Lortie's (1975) statement that 
"cooperation could be extensive outside the classroom but teachers 
preferred to keep the boundaries intact when they actually worked with 
students" (p. 193). Given that teachers receive crucial intrinsic rewards 
from students, teachers may wish to safeguard their student 
interactions, suggesting that team-teaching between teachers may be a 
risky and complex act. Huberman (1993) states that it is difficult for 
two teachers to be responsible for the same students at the same time: 
"The response set of one person would collide, early on, with that of 
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the second, whose reading of the situation and whose rapid, on-line 
responses would necessarily be different..." (pp. 17-18). 

However, many studies have shown that teachers can benefit 
from exposure to new forms of practice with an instructional focus that 
they perceive as meaningful to their students' learning (Grossman & 
Stodolsky, 1994; Mclaughlin& Talbert, 1993; Siskin, 1994; Wasley, 
1992). One approach that might facilitate such an instructional focus in 
teachers' work is their examining teaching and learning situations 
within classrooms — ^the learning of new teaching knowledge within 
context. A contextualized study of teaching can present teachers with 
curriculum in relation to students — ^their responses and learning. In 
discussing this notion of learning "content-in-context," Lieberman 
(1995) writes that "teachers' understanding of student learning and 
development must grow as a result of their continuous inquiry into 
classroom practice" (p. 22). This "experiential learning with learning 
related to the classroom culture" (Lieberman, 1990, p. 532) presents 
teachers with focused instructional inquiry and growth. Related 
approaches include the process approach to teaching writing, whole- 
language learning, cooperative learning, and the Foxfire experience 
(Lieberman, 1990). 

Resources 

Resources play a key role. One seemingly crucial resource is 
time for collaboration (Little, 1993a; Raywid, 1993), which may be 
more important than facilities or even staff development. Raywid 
(1993) calls time the "scarcest resource," needed for teachers to 
observe one another's classes, assess their work, and design 
curriculum, as well as to develop habits of reflection about practice 
(Huberman & Miles, 1984; Schon, 1984). Little (1993a) states that 
teacher growth "calls ...for adequate ‘opportunity to learn' (and 
investigate, experiment, consult, or evaluate) embedded in the routine 
organization of teachers' work" (p. 5). A central feature of resources is 
their ability to build capacity for reflection feedback, and problem 
solving (Fullan & Miles, 1992; McLaughlin, 1993; Lieberman, 1994). 

A Narrative Method 

The study draws from data collected as part of a much more 
extensive ten-year longitudinal study of the recruitment, preparation, 
teaching, induction, and retention of alternate route and college 
prepared teachers (Natriello and Zumwalt, 1992). In the interest of 
space, this article presents only brief narratives of these teachers' 
growth as collaborators. These narratives are then subsequently used 
as the basis for a more analytical discussion of emergent elements of 
support for these teachers. 

A narrative method was selected to allow for the study of 
continuity in the lives of the individual teachers. Both descriptive and 
explanatory narrative (Polkinghome, 1988) were used. In descriptive 
narrative the purpose is "to produce an accurate description of the 
interpretive narrative accounts individuals or groups use to make 
sequences of events in their lives or organizations 
meaningful" (Polkinghome, 1988, pp. 161-162, as cited in Clandinin 
& Connelly, 2000, p. 16). In explanatory narrative, "the interest is to 
account for the connection between events in a causal sense and to 
provide the necessary narrative accounts that supply the 
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connections" (Clandinin & Connelly, 2000, p. 16). A narrative 
approach was used to attempt to capture some of the richness and 
nuances of meaning, as well as ambiguity and dilemma, in human 
affairs (Carter, 1993). Narrative places an emphasis on the connections 
between what humans think, know, and do as well as the reciprocal 
relationship between the way that human thinking shapes behavior and 
knowing shapes thinking" (Behar-Horenstein & Morgan, 1995, p. 

143). 

The study relied primarily upon participants' self-reports of their 
work eind subsequent discussion of narratives constructed from surveys 
and interviews. Participants were presented with four surveys and four 
semi-structured interviews over the first six years of the study and four 
additional semi-structured interviews over the following four years. 
The interviews were the same for both respondents with the exception 
of follow-up probes and prompts. 

In addition to the interviews and surveys, the three teachers were 
given a reconstructed narrative of their history as collaborators in the 
classroom over their first ten years of their teaching lives. Special care 
was taken to ensure that the reconstructed narrative was faithful to the 
teachers' situation and their perceptions of their history. The data in 
these narratives were drawn from existing interview, survey, and 
observational data. These narratives were developed by the researcher 
and presented to the participants initially in written form in advance of 
an in-depth conversation with them about their collaboration history. 
This process allowed participants to examine and reflect on the 
reconstructed narratives before discussing their history as collaborators 
with the researcher. The written and spoken narratives allowed the 
three teachers to check, challenge, and/or contribute to the narrative. 
Through this process, the participants interpreted the data and 
discussed their view of how their collaborative life had been 
composed. 

Constructing the narratives from data as it emerged year by year 
allowed first for the viewing of development as it unfolded, not 
recalled from a distant vantage point filtered through a veil of 
increased experience. The subsequent discussion by the researcher and 
the participants of the reconstructed narratives allowed for a more 
analytical discussion of the events and the meaning of their history as 
collaborators. Thus, narrative was both "phenomena under study and 
method of study" (Clandinin & Connelly, 2000, 4). 

Data analysis 

The data in this study were analyzed in a multi-step process, 
with their reconstruction into narratives as the basis for a discussion 
between the researcher and the participants. Following the discussion 
of the narratives, the data-analysis process was repeated again with the 
subsequent data. 

The data analysis process took the form of a series of 
compressions (Huberman, 1995; Merriam 1988; Yin 1989) in the 
search for patterns (Bernard, 1994). The data first moved from edited 
initial interview, to secondary coding table, to primary coding table. 
The researcher analyzed the data by hand, holding "a conversation 
with the data" (Merriam, 1988, p. 131), in which he jotted down 
general thoughts and reflections and searched for regularities and 
patterns to transform into categories. 

The interviews were organized (or chunked") into "meaning 



httn://enaa.asu.edu/enaa/v9n38 html 



286 



10/29/01 




EPAA Vol. 9 No. 38 Sawyer: Teachers Who Grow As Collaborative Leaders 



Page 7 of 22 



units" and placed on the secondary coding table. Each meaning unit 
was a direct quotation from the interview (Huberman, 1 995). Care was 
taken to maintain data integrity, contextualization, and narrative 
sequence of the responses. Data were, therefore, entered in these tables 
in chronological order in the smallest chunk possible, which still 
provided adequate contextual information. The secondary code (or 
codes) assigned to the meaning unit was then given to each meaning 
unit. These codes used key words from the initial quotation, in essence 
"low inference snippets" (Huberman, 1995), to keep the code as 
faithful to the data as possible. 

The third step in the data analysis process was the assignment of 
the primary codes. The primary codes were developed by grouping 
together and then organizing into patterns and themes the secondary 
codes. The name of an emergent overarching theme would then 
become a primary theme. Following Yin's suggestion that a theoretical 
orientation can guide the analysis (1989), the primary coding tables 
were organized under research question into categories related to 
elements of support found either inside the classroom or outside it. 
Finally, the interviews were read again to identify additional and 
possibly stronger examples of such themes and patterns as well as to 
search for irregularities and contradictory cases (Huberman, 1995; 
Merriam, 1988). 

Contribution 

Few studies have systematically examined how teacher 
collaborators have arranged elements of support over time for their 
collaborative work. Furthermore, few studies have exeunined how 
school culture — either conservative or more progressive — intersect 
over time with independent teachers who go about establishing 
collaborative arrangements for themselves. However, findings from 
the limited number of participates are not offered as a basis for the 
formation of generalizations, but rather as a demonstration of 
plausibility (Behar-Horenstein & Morgan, 1995). As Carter states, 
"Generalizations of this latter form are. . .explanatory propositions with 
which we can make sense of the dilemmas and problematics of 
teaching. (1993, p. 10). The contribution is "intended to be the creation 
of a new sense of meaning and significance with respect to the 
research topic than it is to yield a set of knowledge claims (Clandinin 
and Connelly, 2000, p. 42). 

Unfolding Collaboration 

The following three narratives briefly describe the collaborative 
activity in which Marilyn, a middle-school mathematics teacher, Ellen, 
a high school mathematics teacher, and Susan, a high school English 
teacher, engaged over ten years (including their student teaching). 
These teachers each entered teaching at the same time but worked at 
separate schools and did not know each other. 

Marilyn in Mathematics 

In her first ten years of teaching mathematics on the middle- 
school level, Marilyn engaged in a range of collaborative efforts. Her 
introduction to collaborative work began in her first year of teaching 
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when she herself began to initiate a loosely structured collaboration. 

At this time, Marilyn and three other teachers, including a science 
teacher, organized a ski trip for "at risk" students. Initially, her goals 
were to combine her interests and talents with those of her students, 
while reinforcing her students' learning of mathematics in a real-life 
context. Marilyn envisioned that on this trip her students would at least 
discuss making mathematical applications (e.g., speed, distance, and 
angle problems) as they skied downhill, developing, in the process, 
greater self-esteem, academic motivation, and hands-on 
interdisciplinary knowledge. The other, more experienced teachers, 
however, did not carry-through on their intentions to inject structured 
interdisciplinary study into the fieldtrips. Frustrated, Marilyn alone 
could not have her students make the intended mathematical-science 
applications. 

Marilyn and the science teacher continued to organize and 
sponsor this and other similar trips for the next nine years, though 
dropping their more contextual learning aspects. These fieldtrips 
contributed to her belief that positive social interaction could promote 
students' positive feelings about their class and school. In addition to 
these field trips, starting in her first year (and running through her 
ninth year) Marilyn began having a series of conversations with 
another mathematics teacher about developing new approaches for 
district-mandated proficiencies and tests. 

A second form of collaboration that Marilyn engaged in from 
her second to her sixth years was initiated and structured by her 
school, not her. Examples of this form of collaboration included an 
interdisciplinary teacher-cohort planning team and summer curriculum 
committees. Marilyn found in the cohort situation, to her armoyance, 
the science, English, and social studies teachers banded together and 
did the minimum level of work they thought the administration would 
allow. Starting in her second year, she also engaged in summer 
committees that collaborated to change course proficiency lists, course 
outlines, departmental or district tests, and textbook adoption. Much of 
the work on these committees was susceptible to arbitrary last-minute 
administrative decisions, usually related to inadequate implementation 
of committee work. Marilyn thought that the cohort teams were 
considerably more contrived in that many of the teachers at the school 
had to take part in them. The administration's influence in both of 
these collaborations was centered more on initiation than on follow- 
through. This lack of follow-through played to the advantage of the 
cohort teams, allowing them to disregard administrative program 
goals. Unfortunately, the lack of follow-through was frustrating to the 
teachers on the textbook adoption committees, who wished that the 
administration would support and implement — rather than disregard, 
as they did — their recommendations. Still, these school-initiated 
situations presented Marilyn with relatively structured opportunities to 
learn to share and critique knowledge about teaching and learning with 
her peers. 

Starting in her eighth year, Marilyn began a third form of 
collaboration. Now, both she and the school together initiated and 
structured a collaborative team of teachers to design a new Algebra I 
program to be implemented the following year. While either Marilyn 
or the school initiated the other two collaborations separately, Marilyn 
and her collaborative peers as well as the school jointly initiated and 
structured the Algebra I committee. The school and the teachers 
mutually agreed upon program goals which focused on student 
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learning. In addition, the school gave the team greater autonomy to 
design the program and its follow-up evaluation based on positive 
impact of student learning. As Marilyn began this committee, she 
expressed cautious hope that the school would implement the new 
Algebra I Program as planned. 

Marilyn thought that the collaborative process was successful in 
its goal of establishing an entirely new Algebra I program. Working 
together, the committee devised and followed a clear collaborative 
process. The committee began by raising explicit questions related to 
their knowledge of student learning, content coverage issues, course- 
sequence issues, and the proposed textbooks under review. They then 
evaluated these new textbooks by way of these questions, which were 
drawn from their own practice. One question, for example, was how 
well the books supported students' in-class use of manipulatives, such 
as triangles and scales and supplemental problems. The committee 
then designed a two-year curriculum for the new algebra program. 

Marilyn thought that the committee's success in finding 
consensus was related to its member's camaraderie, as well as similar 
teaching backgrounds and general educational philosophies, within a 
context of mutual respect. At the end of her tenth year, Marilyn was 
waiting with some guarded skepticism to see if the school would 
follow through and implement the committee's recommendations. 

Ellen in Mathematics 

Teaching mathematics on the high school level, Ellen engaged 
in a series of collaborative arrangements in her first ten years in 
teaching. All of these collaborations were relatively conflict free. 
Interestingly, they also followed a pattern that was seemingly 
consistent with how she evolved as a teacher. In the classroom, she 
went from being relatively prescriptive in her first couple of years, to 
more open and experimental in years three though five, to more hands- 
on and experiential after her fifth year. 

Ellen began her first collaboration in her first two years of 
teaching, in which she team-taught a basic math class with a veteran 
teacher. Initiated and supported by the school as an induction program 
for new teachers, this classroom-based collaboration benefited Ellen in 
a number of ways. Ellen and the other teacher had daily classroom 
interaction, daily shared planning and discussion time, and a 
complimentaiy sharing of experience. 

From her third through sixth years in teaching, Ellen's 
collaborative work changed. After teaching algebra for two years, she 
saw a need to change the sequence of mathematics courses, to place a 
beginning geometry course between the first two algebra courses. This 
collaboration was marked by a sense of mutuality of interest between 
Ellen and the school, initiated by both the school and Ellen, organized 
around specific task goals, and, for Ellen at least, relatively focused on 
classroom-based knowledge of student learning. Unlike the type of 
collaborative work beginning in her sixth year, the committee 
outcomes were relatively consistent with Ellen's then current 
approaches to teaching. This collaboration never encouraged her 
explicitly to examine or challenge her assumptions to teaching and 
learning. The following year, the school implemented the sequence of 
mathematics courses in agreement with her recommendations. 

In her sixth year of teaching, Ellen began to realize that the 
district-set midterm exam in geometry was focused on students' basic 
recall knowledge and basic skills. She knew that the test had to be 
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changed, but wasn't sure how. Getting district permission, she and a 
mathematics teacher from the same district but a different school, 
began to plan a new assessment program which facilitated students' 
performance-based learning. 

This coliaborative work was similar to that of the previous ones 
in that it was focused on established course curriculum and allowed 
Ellen to work with people she knew and liked. It differed from 
previous efforts in that Ellen showed much greater initiative and 
experimentation. Also, the process of the collaboration encouraged her 
to challenged and change many of her teaching practices, if not 
teaching beliefs. There was now more clear oversight of the process, 
more conscious experimentation, more recognition of the student- 
context to the assessment format, and more reflection routinized into 
the collaborative process. 

She and her partner approached their collaborative goals by first 
clearly establishing a rationale related both to district goals and 
student-learning considerations. In working together, they focused on 
changing approaches to mathematical format, rather than content. The 
projects they devised for students built on student creativity and 
critical thinking skills related to problem-solving processes more than 
products. Ellen and her partner consciously built oversight and 
reflection into their collaborative process, viewing the first year of the 
new program as a pilot program. In devising their new midterm 
collaboration, Ellen and her partner developed a systematic approach 
to evaluate each other's knowledge of teaching and learning, including 
the use of classroom artifacts and an explicit discussion of how 
students in their classes learn. Sharing a sense of creativity with her 
partner, she and the other teacher began to examine ways of teaching 
that were very different from how they had both taught in the past. 

While a stated district goal of the collaboration was the 
implementation of the new math assessment program, Ellen 
downplayed the importance of greater school or district 
implementation to her feelings of satisfaction with the collaboration 
outcomes. This collaboration ended positively with the school 
implementing and establishing their new assessment program as an 
optional midterm exam. Ellen and her partner made plans to review the 
midterm program in its second year. 

Susan in 'English 

Susan took part in a number of collaborations with her peers. 
While these collaborations initially were somewhat distant from her 
classroom (e.g., a school accreditation evaluation), they eventually 
came to reflect her interests and influence her curriculum. 

In her first two years of teaching, Susan engaged in three school- 
initiated collaborations, activities which every teacher was required to 
join. In one effort, she took part in a Middle States School Evaluation. 
Susan later dismissed this work as obligatory and meaningless. A 
second effort was a cooperative teaching situation in which she 
worked with a writing-lab teacher at her school. She thought the lab 
presented her and her students with an opportunity to change their 
approaches to the writing process, promoting a process of more 
substantive revision to increase the depth of content within their 
compositions. And in the third early collaboration, Susan attended a 
hands-on workshop on "advanced teaching strategies." After attending 
this workshop, she began to design her lessons in relation to the four 
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student learning styles discussed at the workshop. 

In her third year, Susan took a one-year sabbatical in order to 
earn a Master of Arts in Teaching degree at a prestigious Ivy League 
university. Back at her school, she wished to implement her master's 
thesis, an effort which led to her establishing an English/history 
interdisciplinary humanities program with a history teacher. Susan 
initiated this beginning period of collaboration, unlike the earlier one 
that was established by the school. Conceived of as a pilot program, it 
may also be distinguished from the following one by its emphasis on 
reflection and change. 

Before actually beginning to work together, they discussed how 
they would do this and what their collaborative goals were. They 
decided that they would actually team-teach in a blocked class, twice 
as long as a regular class. They also established a daily shared 
preparation time, which let them monitor the class, anticipate 
directions and needs, develop foresight, and reflect on the process. 
While buoyed by a number of successes in this class, their reflection 
focused on perceived issues in the class. After the first year, they 
thought that this approach was too focused on the lockstep chronology 
of history, with the social studies curriculum dominating the English 
curriculum. This reflection led them to add a year-in-review project in 
the second year to make the course less doctrinaire and routine. In this 
project, each student adopted a year as the focal point for a detailed 
project. This project then formed much of the curriculum as the 
students presented it to the rest of the class. 

Susan then took another sabbatical to study writing. When she 
returned to her school, she continued to teach and develop the 
interdisciplinary humanities course. During this year, Susan and the 
history teacher continued to teach and meet together as in the initial 
two-years of the program. They discussed their curriculum in relation 
to a framework which considered teaching-and-learning aspects of 
their course: a desired balance between presenting students with a 
defined course structure and promoting their independence, 
imagination and creativity; and the use of student work to promote 
student creativity and curriculum ownership. 

This time period in her humanities collaboration was marked by 
a number of characteristics. First, she and the history teacher 
established a reflective process which was focused at times on 
relatively nuanced classroom specifics and at times on the way that 
school structures could either support or hinder the humanity course's 
sustainability. For example, they wondered how to promote the 
institutionalization of their program within the school as well as how 
their program could change the culture of their school. Also at this 
time, a conflict arose between Susan and her partner's efforts to 
institutionalize the program and the growing hostility of the school to 
it, creating in Susan's words, a "systemic nightmare" to it. The 
intersection of these situations led them to a decision to terminate the 
program. Ironically, this experience prompted her to realize that to 
become the teacher that she wished to become, she would have to find 
a new school in a new system. As Susan stated in her last year of 
teaching, "You need an entirely new system. . .Ironically, I couldn't 
stay in that system. And I can't go back to that system." 

Teachers' Elements of Support for Collaboration 

The particulars of these three teachers' collaboration with their 
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peers differ. Similar patterns, however, appear in their perceptions of 
support for their collaborative work. 

Personal Characteristics 

Personal characteristics played an important role in how these 
teachers emerged as collaborators. Each of these teachers shared an 
overriding concern for the learning of all of their students. Each 
teacher brought about collaborative situations that reflected personal 
questions about teaching and learning. Each teacher was willing to 
expose her own work to public scrutiny, and each believed (to 
different degrees) in experiential learning. Each teacher thought that 
being able to select her collaborative partners was crucial. By their 
sixth year in teaching, each teacher had developed peer selection 
criteria involving complementary (e.g., educational philosophy and 
views of student learning) yet contrasting (e.g., different approaches to 
practice) elements. And, each teacher thought that a shared philosophy 
of teaching and learning was more important to her collaboration than 
a shared approach to teaching. 

Structured Approaches to Critiques of Practice 

Perhaps the strongest level of support that these teachers found 
to motivate their collaborative work was their awareness that this work 
was directly helping them to improve their classroom instruction for 
all their students. Reluctant to talk about the concept of "teaching 
practice," each of these three teachers preferred instead to discuss more 
specific issues and questions of teaching and learning. A network of 
relationships existed between their evolving views of practice and their 
participation in these collaborations. A scaffolding process appeared to 
be at play in which at different points in their careers there was an 
appropriate balance between support of existing curriculum knowledge 
with positive tension from critiques of practice. Related to how their 
view of support changed over time, these critiques focused in the first 
year or two on preexisting examples of curriculum which they did not 
develop. However, by the third or fourth year, they focused more on 
personal examples of curriculum. This balance may be seen in their 
evolving process of reflection in these collaborative efforts. This 
process of reflection was structured to allow them to critique and 
question forms of practice in ways that became increasingly more 
centered on or more systematically critical of their evolving practice. 
This process is found in the collaborative work of all three cases when 
examined over the course of their teaching careers as a whole. 

Initially, each teacher began to critique and reflect on 
curriculum — but in ways that did not directly expose or threaten her 
own curriculum. They often discussed preexisting curriculum, for 
example changing a course sequence, redesigning district tests, or 
revising existing assignments (e.g., the research paper). A possible 
exception to this pattern may be found in Susan's work with her 
establishment of the humanities program. But initially, this curriculum 
often had a relatively prescriptive, subject matter emphasis. At this 
point, they critiqued less their own practice or views, than curriculum 
that was relatively consistent or similar to it, allowing them to guard 
their still fledgling curriculum making from public inspection while 
possibly examining it by proxy. 

As each of these teachers gained classroom knowledge and 
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expertise, however, much of their reflection on practice consistently 
revolved around questions and dilemmas related to curriculum that 
supported their students as active learners. By their sixth or seventh 
year, all three teachers directly critiqued their own practice in their 
collaborative work, framed by questions that they drew from their 
work. Eventually, each teacher intentionally established collaborative 
frameworks within which to weigh and evaluate multiple approaches 
to curriculum. On a relatively large scale, for example, Susan, 

Marilyn, and Ellen actually established pilot programs to supply 
feedback for subsequent evaluation and revision of their programs, 
again focused on their impact on student learning. These processes 
then allowed the collaborators to share and generate knowledge about 
the same leaning context or environment. And, within this emergent 
context of shared-and-generated knowledge, each of these teachers and 
their partners developed subject content with pronounced process 
elements. 

Evolving Notion of Subject Matter 

Initially these teachers were each relatively traditional in their 
teaching. Over time, however, each teacher's notions of their subject 
matter and disciplines changed. Eventually Ellen, whose later 
collaborative work was confined to geometry, thought that the 
flexibility and relatively open-ended nature of the content of and 
approaches to geometry supported her work with the other teacher. 
Marilyn thought that the relatively fixed nature of the content and 
sequence of mathematics coupled with notions of multiple approaches 
gave her shared ground to discuss algebra with other teachers. Susan 
found that English easily lent itself to an interdisciplinary pairing with 
social studies. She did not collaborate on curriculum related to honors 
English, though, where she may have had a more fixed notion of 
coverage. They all found that criteria for standardizing testing that was 
becoming more open-ended supported their collaborations related to 
curriculum. 

As they developed as collaborators, their approaches to teaching 
were also changing. Over time, they each began to show a tolerance 
for the ambiguity or the multi-layeredness of curriculum, both within 
themselves and between themselves and collaborative partners. In all 
three cases, a growth in pedagogical content knowledge (Shulman, 

1 987) coincided with their synthesizing from their collaboration into 
their curricular planing views of curriculum that they may at one time 
have rejected. 

School Support 

Each of these teachers stressed that support from the 
administration, department, and school was essential to her 
collaboration. Ellen's creative-midterm math program, arguably the 
most sustainable collaboration of these teachers, enjoyed the full 
support of her department, school, and the district (if not peers). 

Susan's remarkable interdisciplinary program, on the other hand, while 
a powerful experience for its students, suffered immensely from a 
hostile administration. These teachers' views of structural supports also 
evolved over the course of their collaborative work with their peers. 
Through their third to fifth year of teaching, they appreciated a greater 
emphasis on the direct contribution of the administration in structuring 
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situations to support their sharing of knowledge and growth of 
curriculum. By about their sixth year of teaching, they each began to 
appreciate support from their administrators for their deeper and more 
personal involvement in their collaborative projects themselves, rather 
than that for more decontextualized innovations in teaching, 
iimovations found, for example, in in-service workshops. A further 
form of administrative support was a shared sense of purpose or 
mission, in which the goals of the collaboration were consistent with 
those of the school and of specific individuals, such as the principal 
and curriculum supervisors. 

These teachers also found that the collaborative process itself 
was supportive to their involvement in collaboration. Wiile no school 
had a coherent program toward collaboration, certain approaches to 
collaboration may have fostered greater teacher involvement in this 
type of work. Ellen, for example, valued her sequence of 
collaborations, going from more-to-less administratively supplied 
structure, undergirded by general school support. These teachers 
increasingly found meaning in collaborative work that allowed them to 
create cycles of growth for themselves. These cycles linked personal 
questions about teaching and learning to peer discussion, 
experimentation, reflection, and the generation of new questions about 
teaching and learning. 

The Evolving Nature of Support 

As these teachers grew in experience and level of reflection, the 
form and amount of support that they viewed as important to their 
collaboration changed. Initially, they each valued collaborative support 
that was more one-on-one and classroom specific. In addition to 
meeting their initial needs as new teachers, this form of support 
facilitated their growth in knowledge and experience in the actual 
process of collaboration. By about their sixth year in teaching, 
however, alt three of the teachers began to seek and value support for 
their collaborative work that was broader and encompassed the school 
as a complex but changeable organization. This latter form of support 
was more systemic and compatible with their growth in knowledge 
about the relationship between meaningful instruction and school 
culture and structures. 

Discussion: Islands of Agency and Initiative 

The unfolding narratives of Marilyn, Ellen, and Susan show the 
unique ways that they developed and acted on personal meaning in 
their work. Their actions at work became increasingly grounded in 
their developing knowledge, questions, and theories about teaching 
and learning. This grounded knowledge informed and was informed by 
the various ways that they constructed curriculum contexts to help 
students learn. They not only persevered in their efforts to work with 
their peers. They also helped to establish greater contexts of support in 
order to collaborate with their peers. In addition, these three teachers 
encountered and challenged — often with considerable personal effort 
— individualism, conservatism, and presentism (Lortie, 1975) inherent 
in school structures. 

Marilyn, Ellen, and Susan's narratives suggest that collaborative 
goals and activities intersected with their school's culture and structure 
and that this intersection became more meaningful for them as they 
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developed greater knowledge and experience from their teaching. As 
these teachers' personal practical knowledge of teaching (Clandinin & 
Connelly, 1995) developed, the relationship between their schools' 
structures of support and their growing personal practical knowledge 
became increasingly important to these teachers. 

Given this backdrop, a question emerges from their narratives: 
Did these teachers develop a sense of initiative and agency for 
personal self-renewal and school change? These questions are 
complicated. First, each of these teachers, at least for a time, did 
develop a growing sense of initiative and agency in her work, both 
within the classroom and within her collaborations. However, the 
degree of self-renewal and satisfaction related to the achieved or 
intended outcomes of this agency was related to their perceived level 
of success in reaching their goals. Ellen, for example, was arguably the 
most successfiil in her collaborative work through her tenth year of 
teaching. Collaborating with the teacher from the neighboring school 
to change the midterm exam in geometry, she changed both district 
guidelines impacting her work as well as the curriculum she made in 
class. In a way, she established an island of agency for herself, 
basically centered on her classroom. Whether on not other teachers in 
her school also changed their midterm exams in response to the new 
guidelines did not directly affect her curriculum changes, which she 
could still carry out. This revision to her curriculum led to an 
invigorating sense of self-renewal for her in her teaching. 

Susan, on the other hand, was much more ambitious in her 
collaborative goals. She initially developed a context to support her 
interdisciplinary course and then established a new program at her 
school. However, the relationship between her program and greater 
school change became increasingly problematic for her. For a time, her 
increased agency and initiative also led to a profound sense of self- 
renewal. Ironically, it also contributed greatly to her leaving the 
classroom to work for the charter school movement to empower 
teachers to start their own schools. 

This difference in the career pathways between Ellen and Susan 
echoes findings of Martin Huberman. In his well-known study about 
the professional life cycle of teachers, Huberman (1989) suggests that 
the teachers in his study experienced multiple career paths at different 
stages in their teaching lives. At the end of a long teaching career, 
some of the teachers in his study were relatively satisfied and content 
with their teaching careers, whereas others experienced a sense of 
frustration and a lack of closure: "Depending on the previous 
trajectory, this final phase can be either serene or 
acrimonious" (Huberman, 1989, p.38). This outcome was partly 
related to the teachers' perception of how successful they had been in 
achieving their goals in teaching. Those teachers who attempted to 
bring about relatively large-scale change were often the most 
dissatisfied when retiring from teaching. 

Restructuring Schools As Sites of Authentic Leadership 

Authentic leadership (Evans, 1 993) values "the head, the heart, 
and the hand" (Sergiovanni, 1992) of leadership and builds from the 
multiple voices and unique strengths found at a given site (Miller and 
O'Shea, 1992). It recognizes that teachers develop and change over the 
course of their careers. This form of leadership is necessary for schools 
to become places of self-regulated learning, not only for students, but 
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also for teachers and other staff members — at different points along a 
teaching continuum from novice to more experienced teacher. While 
teachers who emerge as collaborators and leaders may arrange 
structures of support for themselves in culturally impoverished 
schools, these teachers often pay an emotional and professional price. 
Instead of supporting emergent leadership characteristics in teachers, 
many schools expose teachers to conditions that facilitate contrived 
and superficial forms of collaboration (Hargreaves & Dawe, 1 990). 

There are many ways for schools to value and build from the 
unique voices and strengths of teachers like Marilyn, Ellen, and Susan. 
The following elements might be included in such a consideration. It is 
helpful to recognize that teachers' agency, voice, and sense of meaning 
matter greatly to them as they work with each other and their students. 
Efforts to control the quality of teaching through rigid, centrally 
mandated accountability measures can create sites of contention for 
teachers. 

In addition to personal efficacy, the teachers in this study were 
supported in meaningful collaboration by a dynamic notion of 
curriculum. They each realized that for curriculum to engage their 
students, the students must engage the curriculum. Thus they began to 
view curriculum as a dynamic gestalt of student input, teacher input, 
classroom materials, and inside-as-well-as-outside classroom contexts 
(Clandinin & Connelly, 1992; Sawyer, 1998; Schwab, 1978). This 
more open notion of content calls into question authoritarian views of 
what knowledge is of most worth in the classroom. Given that 
curriculum is a dynamic interaction, teacher support for collaboration 
that involves curriculum will change for each individual teacher at 
different points along his or her career. Furthermore, successful 
collaboration is itself a support for further collaboration as it deepens 
and extends knowledge and expertise about teaching. Schools run the 
risk of losing good teachers by devaluing and dismissing their 
meaningful collaborative efforts. 

Over time and with a growth in teaching experience and 
knowledge, these teachers began to value structural support that 
facilitated their efforts to bring about not only classroom, but also 
program and school change. At least for a time, each of them carved 
out sites of personal growth and renewal, sites which included unique 
support structures. Ellen found professional renewal in change efforts 
that were primarily focused on her classroom. On the other hand, 
Susan's questions about student learning led her to establish an 
interdisciplinary program that bridged classroom walls. The degree to 
which the three schools helped or hindered these two teachers, as well 
as Marilyn, in their quest for the improvement of education for all 
students greatly influenced these teachers' decisions to remain or leave 
the teaching profession. The grounded knowledge that teachers 
generate and share within collaborative islands ought to support the 
predictable success of school reform. 
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Abstract 

Education Weeks report "Quality Counts" judges New 
York State's curriculum and assessment policy efforts to 
be an "A." Surface-level reviews such as "Quality Counts" 
tell something about the workings of state policy, but they 
are more usefol as snapshots than as well-developed 
portraits of curriculum and assessment change. In this 
article, I analyze the new New York State Global History 
and Geography standards and tests using a set of social 
studies-specific criteria which inquire deeply into the 
implications for real instructional change. From that 
vantage, I argue that New York's policy efforts, while 
seemingly well-intentioned and reflective of surface-level 
change, fail to promote powerful teaching and learning in 
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social studies. Teachers intent on producing ambitious 
teaching and learning will find little to interfere with their 
efforts. But as a set of reforms intended to encourage 
substantive change, the new global history test falls short. 



By some reports, New York state has made considerable strides 
in redesigning its state standards and assessment programs. For 
example, the authors of Education Week's report. Quality Counts (see 
http://www.edweek.org/sreports/qcOO) judge New York's efforts to be 
an "A." In that report. New York scored points for having new content 
standards in all school subjects and at elementary, middle, and high 
school levels; for having tests which employ multiple-choice, short 
answer, and extended response questions; for requiring passing state 
assessments for high school graduation; and for using a range of policy 
tools such as report cards, ratings, financial assistance, and state 
sanctions to encourage improved test performance. 

Surface-level reviews such as Quality Counts tell us something 
about the workings of state policy, but they are more useful as 
snapshots than as well-developed portraits of curriculum and 
assessment change. Attempts to construct such portraits demand more 
rigorous criteria than whether a type of test item appears or not. When 
such criteria are applied in the context of the new New York state 
global history exam, it is hard to justify Education Week's lofty grade. 
In short, an A from Education Week isn't enough. 

In this article, I do a document analysis of the new NYS Global 
History and Geography standards and tests using a set of social 
studies-specific criteria which inquire deeply into the implications for 
real instructional change. From that vantage, I argue that New York's 
curriculum and assessment efforts, while seemingly well-intentioned 
and reflective of surface-level change, fail to promote powerful 
teaching and learning in social studies. Teachers intent on producing 
ambitious teaching and learning will find little to interfere with their 
efforts. But as a set of reforms intended to encourage substantive 
change, new global history test falls short. 

Design of the Study 

Led by Patricia Avery from the University of Minnesota, several 
colleagues and I from universities around the U.S. developed a set of 
criteria by which to analyze the new state curriculum and assessments 
emerging in our respective states. (Note 1) Drawing on the current 
thinking in our field, especially as it is reflected in national standards 
documents (e.g.. National Center for History in the Schools, 1994; 
National Council for the Social Studies, 1994) and state-level 
standards (e.g.. New York State Education Department, 1998), we 
constructed criteria that ask to what extent the new state tests ask 
students to: 

• demonstrate knowledge of significant concepts and issues in 
history and the social sciences? 

• consider multiple perspectives on issues and events? 

• manipulate and interpret social science data? 

• engage in higher order thinking about significant social studies 
concepts and issues? 
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I operationalize these criteria in the sections which follow. Note, 
however, that these measures really pose two questions. The first 
inquires about the simple existence of each criteria listed, e.g., is there 
any evidence to suggest that students much demonstrate knowledge of 
significant concepts? The second question implies a quality measure, 
e.g., to what degree must students demonstrate knowledge of 
significant concepts? The first kind of question is not unlike those 
asked by Education Week, where the singular appearance or absence 
of a criteria is deemed important. The second kind of question pushes 
deeper, asking about the importance or meaningfulness of the measure. 
Evidence of a measure is interesting, but the extent to which that 
measure is meaningful seems ultimately more useful. 

The prevailing pattern that emerged from my analysis can be 
termed, "yes, but... ." Yes, there is evidence of attention to the subject- 
specific criteria we developed, but inquiries into that evidence suggest 
that the new global exam comes up far short of a substantive change. 

Background on Nevr York State Curriculum and 
Assessment 

In New York state, the belief that tests drive change is alive and 
well. But while the notion that tests matter is widely held, little 
empirical evidence supports a robust connection between tests and 
learning. In fact. Stake and Rugg (1991) argue that "in sixty years of 
vast international research on school testing, the policy of emphasizing 
test performance in order to improve education has never been 
validated" (p. xx). If true, it is no surprise to learn that the available 
research suggests that the relationship between testing and teachers' 
practices is complicated at best (Cimbricz, in review; Cohen & Barnes, 
1993; Firestone, Mayrowetz, Fairman, 1998; Grant, in press). Tests 
matter to teachers (see, for example. Smith, 1991a, 1991b), but how 
teachers interpret and act on the import of new tests is largely 
uncharted ground. 

That little is known about if and how tests influence teaching 
and learning has yet to inhibit state-level policymakers in New York 
(and most other states) from using them. To understand the recent 
changes in the state assessment program, however, one needs to 
consider the long history of state involvement in curriculum and 
testing. 

New York state policymakers draw on a long history of attempts 
to influence classroom teaching and learning. Administered for over 
100 years, the Regents testing program tests high school students on 
standardized, criterion-referenced exams that are tied to state- 
developed course syllabi in all academic subjects. In social studies, 
students take the Global Studies test at the end of a two-year Global 
Studies course sequence in ninth and tenth grades; eleventh graders 
take the U. S. History and Government test after completing a course 
of the same name. State curricula and tests also exist for elementary 
and middle school teachers and students. 

A Mix of Old and New in New York State Standards and 
Assessments 

The most recent changes in the state curriculum and assessments 
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began in the early 1990s under the previous education commissioner, 
Thomas Sobol. Richard Mills, commissioner since 1994, continued 
that effort. Interestingly enough. Mills came to New York intending to 
decrease the traditional emphasis on standardized testing. The 
education reform movement Mills led in Vermont resulted in a state- 
level assessment program based on student portfolios rather than on 
tests. Mills abandoned this approach in New York, however. Sensing 
that the state's draft curriculum frameworks were being largely 
ignored. Mills reportedly asked a teacher to explain. "'You don't get it,' 
the teacher said, with what Mr. Mills remembers as almost a sneer. 'If 
the standards are not on the test, they're not real'" (Hartocollis, 1999, 
Bl).(Note2) 

That comment apparently proved key for Mills is now an 
unabashed supporter of standards-based tests as a vehicle for 
classroom change. The Learning Standards for Social Studies (New 
York State Education Department, 1998) represent the state's latest K- 
12 curriculum; new tests in grades 5, 8, 10, and 1 1 are emerging over 
the next two years. 

Compared with the previous round of curricular revisions in the 
mid-to-late 1980s, the new standards documents represent a mix of old 
and new. Virtually no change appears in the K-5 grades curricula, 
which continue to follow an expanding horizons model. There are also 
no discernible changes in the seventh and eighth grade U.S. and New 
York State history sequence, or in the twelfth grade Participation in 
Government and Economics courses. A modest change is evident in 
the eleventh grade U.S. history and government course in that a 
emphasis on geography surfaces. Major changes seem localized at 
sixth grade, where the course of study expands from Western and 
Eastern Europe and the Middle East to the entire Eastern hemisphere, 
and at ninth and tenth grades, where the emphasis has changed from a 
cultural approach as represented in Global Studies to a chronological, 
history-based study expressed as Global History and Geography. 

The state-level testing program also reflects a mix of old and 
new. Compared to the tests in most other subjects, the new social 
studies assessments seem the least changed. Whereas the new 
mathematics, science, and English-language arts tests make liberal use 
of open-ended and extended tasks, the social studies exams continue to 
rely largely on multiple choice questions. Moreover, compared to the 
tests in sister subject matters, the multiple choice questions posed on 
the social studies exams seem directed toward lower levels of 
understanding. 

The multiple choice questions notwithstanding, the new state 
social studies exams do look different from the old ones. The principal 
change is in the writing portion of the exam. Unlike many minimum 
competency tests. New York students have always had to write essays 
on state exams. The new tests are different primarily in the fact that a) 
students will no longer have a range of essay prompts to choose from, 
and b) a new kind of essay question, a document-based question 
(DBQ), is being introduced on each of the fifth, eighth, tenth, and 
eleventh grade tests. (Note 3) A DBQ asks students to write an essay 
synthesizing information from as many as eight primary source 
documents (e.g., short quotes from government documents and famous 
individuals, political cartoons, poems, charts and graphs). The DBQ 
from the Global History and Geography exam administered in June, 
2000 is as follows: 
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Historical context : Economic systems attempt to 
meet the needs of the people. Capitalism and 
communism represent two different ways to meet 
people's economic needs. 

Task: Using information from the documents and 
your knowledge of global history, answer the 
questions that follow each document in Part A. Your 
answers to the questions will help you write the Part 
B essay, in which you will be asked to: 

Describe how these two economic systems attempt to 
meet the needs of the people 

Evaluate how successful each system has been at 
meeting the economic needs of the people 



This task is followed by eight documents, seven quotations (e.g., 
R.W. Emerson, Adam Smith, Friedrich Engels) and one political 
cartoon, which present contrasting views of communism and 
capitalism. One or two main idea questions accompany each 
document. An example of a document and the attendant question 
follows: 



...masses of laborers. . .crowded into 
factories. They are slaves of the machine 
and the manufacturer. Instead of rising 
as industry progresses, they sink deeper 
and deeper into poverty. . . Karl Marx 
and Friedrich Engels, The Communist 
Manifesto 

The attendant main-idea question is: 
"According to Marx and Engels, what was the effect 
of the capitalist factory system?" 

After responding to short answer questions 
such as this, students are directed to: 

• write a well-organized essay that includes an 
introduction, several paragraphs, and a 
conclusion 

• use evidence from at least four documents to 
support your response 

• include additional related information. 



High school students will also write a second, "thematic" essay 
based on a single prompt. The thematic essay from the June, 2000 
Global History and Geography exam is: 
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Write a well-organized essay that includes an 
introduction, several paragraphs addressing the task 
below, and a conclusion. 

Theme: Justice and Human Rights— Through history, 
the human rights of certain groups of people have 
been violated. Efforts have been made to address 
these violations. 

Task: 

• Define the term "human rights" 

• Identify two examples of human rights 
violations that have occurred in a specific time 
and place 

• Describe the causes of these human rights 
violations 

• For one of the violations identified, discuss one 
specific effort that was made or is being made 
to deal with the violation. 

Students are then advised; 

You may use any example from your study of global 
history. Do not use the United States in your answer. 
Some suggestions you might wish to consider 
include: Christians in the early Roman Empire, 
native peoples in Spain's American colonies, 
untouchables in India, blacks in South Africa, Jews 
in Nazi Germany, Muslims in Bosnia, Kurds in Iraq 
or Turkey, or Tibetans in China. 



Each of the two essays is scored by two classroom teachers on a 
6 point rubric, from 0-5. On the DBQ above, a score of 0 "fails to 
address the task or theme, is illegible, or is a blank paper." By contrast, 
a score of 5: 

• addresses all aspects of the task by accurately analyzing and 
interpreting at least four documents 

• thoroughly describes and evaluates capitalism and commimism 

• incorporates information from the documents in the body of the 
essay and may cite from the document in an appropriate fashion, 
but does not copy the entire document 

• incorporates relevant outside information such as the early 
British factory system, Stalin's five-year plans, collapse of 
communist system in the Soviet Union 

• takes into account the point of view of the authors in the 
description and evaluation of capitalism and communism 

• is a well-developed essay, consistently demonstrating a logical 
and clear plan of organization 

• introduces the theme by establishing a framework that is beyond 
a simple restatement of the task or historical context and 
concludes with a summation of the theme 



httn ://enaa. asu.edu/eDaa/v9n3 9.html 



308 



10/29/01 





EPAA Vol. 9 No. 3...: Analyzing the New York State Global History and Geography Exa Page 7 of 22 



Scores between a 5 and a 0 reflect lesser attention to each of the 
points above. For example, under a score of 3, the third point states, 
"incorporates limited or no relevant outside information." 

Once the tests have been corrected, teachers are directed to a 
conversion table on the back cover of their manuals. There, they total a 
student’s multiple choice and short answer scores (total of 61 possible 
points) and then look across a series of columns from 0-10, which 
represent the least to most possible points on the two essays. At the 
cross-section of these two scores is a converted score which ranges 
from 0-100. In the past, students had to score a 65 in order to pass the 
exam. A 65 is still the targeted state score, but districts are allowed to 
lower the required passing score to 55 for the next couple of years. 

If the new tests themselves are only modestly revised, two other 
changes seem more dramatic. One is that the new fifth and eighth 
grade tests will mirror the high school exams in form and will produce 
individual student scores. Previously, tests at those levels, termed 
"Program Evaluation Tests," were general knowledge exams aimed at 
helping teachers understand the effectiveness of their content and 
pedagogical decisions. The shift to Regents-like tests and individual 
student scores at lower grades seems intended to raise the stakes of 
these tests by tying them more directly to the high school Regents 
exams. The second change concerns the function of the Regents test. 

In the past, passing Regents tests in all academic subjects meant that a 
student earned a "Regents" diploma, a distinction of some note. 
Students who desired to could opt to take the less rigorous Regents 
Competency Exam (RCT) and earn a local diploma. Beginning in 
2001, ninth graders will no longer have these options. The RCT is 
being phased out, and all students will have to pass five Regents 
examinations (English, mathematics, global history, U.S. history, and 
science) in order to graduate. 

"Yes, But... Analyzing the NYS Global History Exam 

By most any measure, NYS policymakers deserve credit for the 
curriculum and assessment revisions they have made. They might have 
taken a less ambitious route by leaving the state curriculum and tests 
largely unchanged or by reverting to a minimal competency exam. 
Since they did not, Education Week's grade of A may well be justified. 
But if the criteria applied are more rigorous and more specific to the 
subject matter of social studies than those used by Education Week, 
then other interpretations of the new standards and assessments seem 
valid. 

Recall that I analyzed the NYS Global History and Geography 
exam by asking to what extent the new state tests ask students to: 

• demonstrate knowledge of significant concepts and issues in 
history and the social sciences? 

• consider multiple perspectives on issues and events? 

• manipulate and interpret social science data? 

• engage in higher order thinking about significant social studies 
concepts and issues? 

Recall also that 1 split this question in two. First, I looked for the 
mere existence of each criteria. Second, I inquired about the quality of 
the evidence for each criteria. My ^lnalysis suggests that while 
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evidence for each of the criteria can be found, in no case is the quality 
or meaningfulness of that evidence strong. In short, the answer to each 
question is, "yes, but... ." 

Knowledge of Significant Concepts and Issues 

To be sure, there is a whole lot of knowledge represented on the 
new global history exam. This claim prompts little surprise, however, 
given the scope of the course title (i.e., "Global history and 
geography"), the 27 single-spaced pages of the state curriculum, and 
the fact that the course is taught over two school years. A quick review 
of the curriculum and test suggests apparent attention to significant 
concepts and issues: Geographic influences, religious beliefs, 
economic systems, political forces, cultural practices, and international 
relations map across an array of developed and developing, ancient 
and modem civilizations. 

Yet even a surface-level analysis begins to yield some problems. 
For while the testmakers develop items for a wide range of concepts 
and issues, a quick count of the multiple-choice questions offers some 
troubling patterns. One pattern is that questions related to western 
nations (i.e., Europe, including Russia/USSR) dominate the test: 
Twenty-four questions assess issues relevant to the west, while only 10 
questions each are assigned to India/Asia and to the rest of the world 
(Africa, Latin America, Caribbean, South America, and the Middle 
East). (Note 4) A second pattern is that the numbers of questions 
related to early civilizations (8 questions) and the middle ages (9 
questions) are notably subservient to those attached to the modem era 
(3 1 questions). (Note 5) 

This latter pattern could be predicted for two reasons. One is that 
historians and social scientists know more about modem times than the 
past, so to see that tmism reflected in the apportionment of test 
questions is no surprise. The second reason is that the state curriculum 
gives preference to the modem era (18 pages) over early (6 pages) and 
middle (3 pages) periods. Since the test is reputed to reflect the state 
social studies standards, it makes sense that the ratio of questions 
would reflect the chronological preferences established in the 
curriculum. 

The first pattern is harder to understand, however. First, the clear 
preference for western-based questions flies in the face of the national 
movement to be more inclusive of other cultures. While the debate 
over multiculturalism has been contentious, it is hard to understand 
why the testmakers would so clearly privilege western history. This 
action is also hard to understand from a curricular point of view. While 
New York policymakers' efforts at creating a multicultural curriculum 
have been variously praised (Combleth & Waugh, 1995) and 
excoriated (MacDonald, 1992), the rhetoric in the social studies 
standards appears to support a strong endorsement of a global 
perspective: 

This curriculum provides students with the opportunity to 
explore what is happening in various regions and civilizations at a 
given time. In addition, it enables them to investigate issues and 
themes from multiple perspectives and make global connections and 
linkages that lead to in-depth understanding. (New York State 
Education Department, 1998, p. 71) 

The decision to emphasize questions related to the west is 
especially difficult to defend when one realizes that within each of the 
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chronological units described in the state standards is attention to 
western and non-western people, events, and issues. For example, the 
unit entitled, "Global Interactions (1200-1650)," is divided into four 
sections, two of which— Early Japanese History and Feudalism and The 
Rise and Fall of the Mongols and Their Impact on Eurasia— are 
explicitly non-western. (Note 6) European issues and events do 
dominate the later units as world and cold wars get heavy play. That 
said, on the relationship between the west and the rest of the world, the 
disparity between the state standards and the state test is stark. 

The disparities between the nations and eras represented and 
between the state curriculum and exam are interesting, but really do 
not help us understand whether the concepts and issues portrayed are 
significant. But then what constitutes a significant event turns out to be 
a pretty thorny issue, both for historians (see, for example, Carr, 1961) 
and for students (Barton & Levstik, 1997, 1998; Grant, 2001; Seixas, 
1994, 1997). One might debate the relative merits of questions related 
to Karl Marx v. Ho Chi Minh, but it seems that with few exceptions 
the test addresses the big ticket items of a standard account of global 
history. 

And that's part of the problem. The disparity in questions 
between western and non-western nations notwithstanding, the real 
issue related to significance is the type of questions asked rather than 
the content. In short, test makers aimed at low-level knowledge 
questions rather than at higher-order thinking questions. As a case in 
point, consider this multiple-choice question: 



The Magna Carta, the Glorious Revolution, and the 
writings of John Locke all contributed to Great 
Britain's development of: 

1 . absolute monarchy 

2. ethnic rivalries 

3. parliamentary democracy 

4. imperialist policies 



Typical of the multiple-choice section, this question reflects an 
emphasis on generally expected, and clearly western constructs and 
events. But while the significance of these elements to global history is 
undeniable, the question merely asks students to identify and label 
them. I address this notion of insignificant questions about significant 
events more directly in succeeding sections. For now then, my analysis 
suggests that, yes, the new state global exam does demonstrate 
attention to important concepts and issues, but does so in a way that 
may not push students' thinking. 

Multiple Perspectives 

The criterion of the extent to which the new state test addresses 
multiple perspectives is another case of "yes, but... ." While the 
inclusion of the DBQ indicates a move toward multiple views, that 
move is less apparent in the multiple ehoice section than one might 
expect. Moreover, the heavy tilt toward western themes undercuts the 
range of perspectives possible. 
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Several multiple-choice questions appear to reflect diverse 
perspectives because they give students multiple pieces of information. 
On closer inspection, however, all but two questions present 
compatible rather than diverse viewpoints. Typical of this kind of 
question is the following: 



Base your answer to question 1 0 on the statements 
below and on your knowledge of social studies. 

• Statement A \ The might of a country consists 
of gaining surpluses of gold and silver. 

• Statement B: A nation's strength is found in 
economic independence and the maintenance 
of a favorable balance of trade. 

• Statement C: We need to gain colonies both a 
sources for raw materials and as markets for 
our manufactured goods. 

Which economic system is being described by 
these statements? 

1 . traditional 

2. feudal 

3. command 

4. mercantile 



Students read three different statements, but each statement is 
necessarily tied to the others as a vehicle for defining mercantilism. 
Rather than dealing with multiple perspectives, then, students must 
only deal with multiple pieces of information. (Note 7) 

The two multiple choice questions which do ask students to 
untangle multiple views employ the same stem: 



Base your answers to questions 46 and 47 on the 
speakers' statements below and on your knowledge of 
social studies: 

• Speaker A\ The gods approached Vishnu, the 
lord of creatures, and said: "Indicate to us that 
one person among mortals who alone is worthy 
of the highest rank... " Vishnu reflected, and 
brought forth a glorious son who became the 
first king. 

• Speaker B: The traditional African society, 
whether it had a chief or not, was a society of 
equals and it conducted its business through 
discussion. 

• Speaker C: Ideally, the best form of 

- government is one where every citizen not only 
has a voice, but also, at least occasionally, is 
called on to take actual part. 

• Speaker D: A monarch's authority comes 
directly from God, and this is how the 
leadership and power in a society should be 
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determined. 

46. Which speakers would support the theory 
explaining the power of France's Louis XIV, Spain's 
Philip II, and England's Elizabeth I? 

1 . A and D 

2. B and C 

3. A and C 

4. B and D 

47. Which speakers would agree with the idea that 
some form of democracy is the best way to govern a 
society? 

1 . A and D 

2. BandC 

3. AandC 

4. B and D 



One could quibble with the fact that the four statements 
represent only two views of government, but that really is a quibble. 
The questions might have been worded more clearly (especially #46), 
but the point remains: Students must be able to sort through differing 
views of government in order to make sense of the questions posed. 

What seems like a similar quibble above rises to the level of 
critique in the DBQ. The eight documents divide cleanly into four 
categories: those that support capitalism (an excerpt from an 
unidentified work by Ralph Waldo Emerson; an excerpt from Adam 
Smith's Wealth of Nations), those that support communism (a quote 
attributed to "Katia," a 16-year-old ninth grader from Moscow in the 
1980s; an excerpt from Friedrich Engels, Principles of Communism', an 
excerpt from Harry Schwartz in The New York Times, 1952), those that 
critique communism (an excerpt from, "The Peasant Wars on the 
Kremlin," by T. P. Whitney; a political cartoon from the Providence 
Journal Bulletin), and those that critique capitalism (an excerpt from 
The Communist Manifesto by Marx and Engels). (Note 8) Those who 
would question the documents selected would rightfully emphasize the 
clean lines of support for and critique of each system. There is no gray 
area here, for depending on the source, capitalism and communism are 
either portrayed as sin or salvation. In the first part of their essays, 
students are asked merely to describe how each system attempts to 
meet its citizens' needs. Since five of the eight documents provide 
clear fodder for this task, it hardly seems a significant challenge. The 
second task, to evaluate how successful each system has been in 
meeting its citizenry's needs, seems more cognitively provocative. 
Here, students would presumably draw on the documents which 
critique each system. But notice what is missing: Students are 
provided only with partisan critiques. No data appear, for example, on 
how citizen-workers have fared under the respective systems. 
Presumably, students will draw on their knowledge that some 
countries like the former Soviet Union have renounced communism. 
But without more and better data, and especially data that offers direct 
comparisons, it is difficult to see how students can do much with this 
task. 
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Those who would defend this DBQ might counter that even a 
weakly constructed DBQ offers a profoundly different task than 
students normally undertake on a standardized test. That so much of 
the testing in social studies relies on multiple-choice questions has 
long been a sore spot among social studies educators. Clearly, this 
DBQ offers a new opportunity for assessing students' knowledge and 
skills. 

Taken together, these points underscore the "yes, but..." 
argument about the new global exam. Including a DBQ ratchets up the 
substance of the test and begins to promote the notion of multiple 
perspectives. But the strength of that claim is undercut by what, with 
seemingly little effort, could have been a more powerful experience. 
Substituting documents that presented more nuanced views of 
capitalism and communism and that presented some comparative data 
would have gone considerable distance in beefing up a fledgling effort. 

Manipulating and Interpreting Social Science Data 

As noted above, the authors of the DBQ could have enhanced 
the student tasks by including some comparative data. Doing so would 
have contributed greatly to the generally weak way that social science 
data are handled on the global exam. 

The types of questions represented on the new test generally call 
for definitions of terms (e.g., limited monarchy, totalitarianism, 
NAFTA) and identification of people, events, and social trends (e.g., 
Napoleon Bonaparte, French Revolution, democracy in Latin 
America). Few questions probe much below a surface-level knowledge 
of global history. And of those questions, a mere handful deal with 
social science data. To be sure, there are questions which employ 
illustrations, political cartoons, and maps. None of these, however, 
qualifies as data in the sense that students are presented with 
information that they must manipulate and interpret in order to answer 
the attendant questions. Of the 50 multiple choice questions, then, only 
three call upon students to use data. One question presents students 
with two circle graphs of the world population by region. The first 
graph shows the distribution for Europe, China, Latin America, North 
America, India, and four other areas in mid- 1992; the second graph 
projects the distribution for the same regions in 2025. Two questions 
follow; 



Which factor best explains the projected change in 
China's population by 2025? 

1 . increased immigration to China 

2. religious doctrines discouraging birth control 

3. government limits on family size 

4. increased agricultural production in China 

Which conclusion about world population in the next 
25 years is best supported by the information in these 
charts? 

1 . Technological improvements will cause a 
population decline throughout Asia. 

2. Developed nations will be home to a majority 
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of the world's population. 

3. Efforts to curb population growth in 
developing nations will be successful. 

4. Africa may experience problems with 
overpopulation. 



A few questions later, students encounter a chart describing 
Internet usage in countries across the world. Three categories of usage 
along with representative countries are listed. For example, "heavy 
usage" countries include Canada, Norway, and the United States; 
"medium usage" countries include Chile, Britain, and Argentina; and 
"little use" countries include Mexico, Columbia, and Saudi Arabia. 
One question follows: 



Which conclusion about Internet usage can be drawn 
from this chart? 

1 . Developing nations have easier access to the 
Internet than developed nations do. 

2. A high standard of living in a nation is linked 
to high Internet usage. 

3. Internet usage limits international cooperation. 

4. Eastern Hemisphere nations use Internet 
connections more than Western Hemisphere 
nations do. 



The final data-based question features a web diagram of 
automobile production using straight lines and arrows illustrating the 
global connections between auto companies and the countries in which 
they originate. For example, Chrysler/USA is connected by a straight 
line to Renault/France and to Hyundai/South Korea and by an arrow to 
Mitsubishi/Japan. The distinction between straight lines and arrows is 
not explained. One question follows: 



Which conclusion can be drawn about global 
economics in the 1990s? 

1 . Countries became more economically isolated. 

2. Higher tariffs reduced trade between nations. 

3. France dominated the world automobile 
industry. 

4. Economies of the world were increasingly 
interdependent. 



These questions meet the ostensible parameters of data-based 
situations: Students are presented with some data from which they 
must infer trends. That said, there are at least two problems with these 
questions. One problem is that students need not manipulate any of the 
data in order to answer the attendant questions. Students must make an 
interpretation, but in all cases the "right" answers are fairly obvious. 
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The reason the answers are so obvious speaks to a second 
problem: Not one of the questions demands much in the way of prior 
social studies knowledge. In short, the questions are cast such that only 
one answer makes common sense. Consider just two examples. First, 
without knowing anything about China, the question about the 
projected change in its population can only be reasonably answered 
with response #3 since it is the only answer which explains a declining 
populace. The answer to the question about global economics is just as 
commonsensical. The only response consistent with the web-like 
diagram is #4 which features the language of "increasingly 
interdependent." (Note 9) True, students need to know the vocabulary 
used— population, interdependent, and the like— but these are hardly 
arcane words used only in social studies contexts. So while students 
are asked to make inferences from the information presented, not only 
are they low-level inferences at best, but the possible answers are 
phrased such that the answers are obvious. 

Once again, then, the surface-level qualifications of the NYS 
global exam pass muster, but a peek below that surfaces undercuts any 
confidence in the A grade assigned by Education Week. As with each 
of the preceding criteria, the new test fails to push students' thinking in 
substantive directions. The appearance of asking students to 
manipulate and interpret data is not enough. 

Higher Order Thinking 

It is probably clear by now that this last criteria, the extent to 
which the new test asks students to engage in higher order thinking 
about significant social studies concepts and issues, lies at the heart of 
my critique. The test makers can legitimately claim some attention to 
each of the preceding criteria. On the level of that attention, however, 
reasonable objections can be lodged. I will not speculate as to why the 
exam was constructed in this manner, but that it came so close to being 
a rich experience for students only to fail, is discouraging. 

Consider two examples of how the exam questions might have 
been enriched. I argue above that the DBQ is composed entirely on 
partisan views of capitalism and communism. A small, but significant 
improvement would be to substitute a graph offering descriptive data 
on the comparative economic productivity and/or the social service 
conditions of the two nations. Such an addition would not only expand 
the range of documents students consider, but it would also help them 
make a more reasoned response to the portion of the essay prompt that 
calls for them to "evaluate how successful each system has been at 
meeting the economic needs of the people." 

The multiple choice questions might also have been improved. 
Consider this example from the 1 994 NAEP Geography Assessment 
(National Assessment of Educational Progress, 1994): 

Statistical Comparison of Two Countries 





Country A 


Country B 


Total Population 


7,193,000 


123,120,000 


Urban-Rural 

Urban 


49.0% 


76.7% 


Rural 


51.0% 


23.3% 


Religions 


Rom. Cath.:92.5% 


Shinto*: 89.5% 


Baha'i: 2.6% 


Buddhist*: 
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Other; 4.9% 


76.4% 

Christian: 1.2% 
Other: 9.3% 


Life Expectancy at 
Birth (years) 

Male 


50.9 


75.9 


Female 


55.4 


82.1 


Age Distribution 
Under 15 


43.4% 


19.0% 


15-29 


26.4% 


21.6% 


30-44 


15.7% 


22.4% 


45-59 


9.3% 


20.1% 


60-74 


4.4% 


9.2% 


over 74 


0.8% 


7.7% 


% of the Population 
over 25 with 


48.6% 


0.3% 


No Formal Schooling 






Leading Exports 


Natural Gas: 21.0% 


Motor Vehicles: 


(as % of total exports) 


Tin: 12.0% 


18.4% 


Zinc: 5.7% 


Machinery: 




Silver: 5.6% 


10.9% 




Antimony: 4.0% 


Iron and Steel: 




Coffee: 2.0% 


5.8% 




Sugar: 1.5% 


Chemicals: 5.3% 




Hides: 1.4% 


Textiles; 2.6% 
Vessels: 1.5% 
Radios: 0.8% 
Televisions: 
0.7% 






(*Some persons 
practice both 
religions.) 



Which of the following statements most accurately 
describes Country A? 

• A. It is dependent on raw material exports. 

• B. It probably has a high literacy rate. 

• C. It has a predominantly urban population 

• D. It will experience slow population growth. 

Which of the following statements most accurately 
describes Country B? 

• A. It has few medical facilities. 

• B. It is industrialized. 

• C. Its primary imports are manufactured goods. 

• D. Its population is primarily employed in 
agriculture. 
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Country B is most likely 

■ A. Botswana 

■ B. India 

■ C. Ireland 

■ D. Japan 



Like the questions on the NYS global exam, these examples 
require understanding of significant social studies terminology. Unlike 
those questions, however, these examples push students to do more 
than define those terms. The first two questions demand that students 
evaluate the data in each cell and to draw conclusions across those 
cells. For example, in the first two questions, each of the possible 
answers directs students toward at a different cell of data. To select the 
best answer, then, a student must evaluate the information across the 
chart. The third question also asks students to look across multiple 
cells, but it adds a twist: Students must compare their assessment of 
Country B's attributes with their previous knowledge of world 
countries in order to select the best response. 

These two brief examples point to the possibilities missed on the 
current exam. It seems safe to say that the bulk of the new global exam 
aims at low-level knowledge and understanding. The majority of the 
objective questions call for defining terminology, identifying 
significant people, places, and events, and in the case of the short 
answer section, describing the main point of a document. Surprisingly, 
the essays push no harder. The thematic essay asks students to 
complete several tasks, but by giving the students numerous examples 
of human rights cases, it is difficult to imagine many students 
struggling. The DBQ seems similarly poised. Students must synthesize 
the views from eight different documents, but there is no nuance in any 
of them and the clean divisions among them play directly into the tasks 
to which students are assigned. Taken together, the array of questions 
on this exam promise much. They do not deliver. 

There is one more dimension that is worth note under the 
general criteria of higher order thinking. The NYS exam presumably 
scores high on the Education Week criteria in part because of the 
"extended response" items or essays. Moreover, the DBQ seems 
designed to signal a change in the structure of the social studies exams: 
One might argue that such a question represents a major shift away 
from traditional testing and toward more authentic assessment of 
students' historical understanding and reasoning. The scoring guide for 
the test, however, mitigates that claim: In short, students can easily 
pass the test without a single DBQ point. In fact, students can pass the 
exam without any essay points at all. A conversion chart on the last 
page of the teacher guidelines indicates that if students total a 
minimum of 54 points from the total of 6 1 possible multiple choice 
and short answer questions, they pass with a converted score of 65 
regardless of whether or not they even attempt the essays. (Note 10) In 
this light, one can argue that the written portion of the new test has 
been substantially discounted compared to the previous exams. Where 
the essays once counted for 45% of a student's score, they now account 
for only 29%. Thus students can leave the essays blank, answer 
correctly approximately 72% of the multiple choice and short answer 
questions, and still pass the exam. Adding the DBQ, then, can be read 
as a minor revision at best. 
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Implications 

Since the mid-1990s, state policymakers have introduced a 
number of curriculum reforms such as new state standards for social 
studies. Preliminary indications (Grant, Derme, Gradwell, Lauricella, 
Pullano, & Tzetzo, 2000) suggest, however, that NYS global teachers 
view the curriculum and assessment changes as a mixed bag. Some 
applaud the state's move to a chronological approach as a more 
coherently historical mode. Others condemn this move (and some 
individual teachers and whole departments have rejected it) arguing 
that it undercuts the power of a cultural studies approach. 

More important than the curricular changes, however, are 
teachers' concern about the new state tests (Grant, 1997a, 2000). This 
makes sense for two reasons. First, the curriculum documents 
produced thus far offer teachers little assistance in making concrete 
instructional decisions (Grant, 1997b). Second, the messages teachers 
receive often promote the view that tests are intended to drive change 
(Grant, 1996). For example, during sessions introducing the new state 
social studies standards, one representative from the New York State 
Education Department said that new tests will "help grow change in 
the system." During another session, a different SED representative 
said, "New assessments will represent a change in instruction.. ..Kids 
won't perform well until (teachers') instruction reflects this." And at 
yet a third meeting, NYS Commissioner Richard Mills added, 
"Instruction won't change until the tests change." The message that 
tests matter also surfaced during local school and district meetings. A 
suburban district social studies supervisor, for example, told teachers 
that "change in content will come if we change the tests." An urban 
district supervisor observed, "If we change the assessments, we'll 
change instruction" (Grant, 1996, p. 271). One might question the 
focus of test influence— instruction, curriculum, or the "system" in 
general— but it is hard to miss the larger point: tests matter. 

But how the new tests will matter deserves continued 
investigation. Our initial work in this area (Grant, Derme, Gradwell, 
Lauricella, Pullano, & Tzetzo, 2000) suggests that teachers' views of 
the new tests reflect some ambiance. Most teachers support the use of 
documents and the DBQ. Yet from what teachers have seen in the test 
sampler disseminated by the state education department, few see this 
move as necessitating a fundamental shift either in their own 
pedagogies or as indicating a fundamental shift in the state's emphasis 
on social studies knowledge as represented in multiple-choice 
questions. 

The analysis above, which focuses on the first test administered 
to NYS tenth graders last spring, suggests that teachers have it about 
right: The new test represents little in the way of fundamental change, 
and so can be read as demanding little change in classroom practices. 
True, some teachers report a ratcheting up of anxieties by students, 
parents, and administrators as test scores become media fodder. But 
responding to test score concerns and responding to the tests at hand 
may be two very different things. 

Notes 

1 . The impetus for this action was a symposium entitled, "State 

Standards-Based Assessments and the Social Studies" held 
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during the annual conference of the National Council for the 
Social Studies, San Antonio, Texas in November, 2000. Pat 
Avery and I were joined by Robin Chandler (Kentucky), Jean 
Craven (New Mexico), and Ceola Ross Baber (North Carolina). 

2. Thanks to Sandra Cimbricz for bringing this quote to my 
attention. 

3. Mock test items, called test samplers, are available for the grades 
5, 8, 10, and 1 1 tests (see 

http://www.emsc.nysed.gov/ciai/assess.html). The first 
administration of the grade 5 test is scheduled for November 
2001; the new grade 8 and 1 1 tests are scheduled for June 2001. 

4. Seven additional questions lump together people, places, and 
events such that it is difficult to ascribe them to a category. 

5. Three additional questions span these time ranges and thus are 
difficult to categorize. 

6. The other two segments are: The Resurgence of Renaissance 
Europe and Global Trade and Interactions. 

7. Alert readers will note that Statement C is the key to the correct 
answer. Students might consider Statements A and B, but these 
are general features of most economic systems. 

8. The quote from "Katia" might be double-counted as both in 
support of communism and in opposition to capitalism in that, 
before the bulk of the quote which does the former, she offers 
this presumed critique of capitalism: "Capitalists are rich people 
who own factories and have lots of money and workers." 

9. I recognize that the adverb "increasingly" is problematic since 
no comparative data is presented. Nevertheless, in testmakers' 
parlance, it is clearly the "best answer." 

1 0. Even more startling is the fact that in those many districts that 
opted to lower the passing score to 55, students need only get 44 
of the possible 61 points to pass. 
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Abstract 

The past decade has occasioned a dramatic increase in 
research on relationships between school size and a 
variety of outcomes, including measured achievement, 
high school completion rates, and postsecondary 
enrollment rates. An interesting interaction effect which 
has been found in replications across seven very different 
states is that as school size increases, the "achievement 
test score costs" associated with the proportion of 
economically disadvantaged students enrolled in a school 
also increase. In short, as schools get larger, average 
achievement among schools enrolling larger proportions 
of low socioeconomic-status students suffers. A 
traditional argument against smaller schools, however, is 
that they are simply too expensive to operate (regardless 
of proven benefits). Large consolidated schools—often 
with narrowly specialized grade spans— are typically 
proposed and constructed as necessary to "save money" 
and to meet the "developmental needs" of certain age 
groupings. This article has two objectives. First, to 
determine if the size-by-socioeconomic status interaction 
effect proves robust across alternative regression model 
specifications, as it did across differing states. Second, to 
make a tentative judgment as to whether the equity gains 
associated with smaller schools are incompatible with the 
need for fiscal efficiency. The analyses (based on our 
Texas data set) suggest that the answer to the first 
question is "yes" and the answer to the second question is 
"no." In particular, the K-12 "unit school" configuration in 
Texas is shown to be both educationally effective and cost 
effective. 



Educational researchers and policy makers rarely meet an issue 
they are willing to resolve once and for all. School size is a case in 
point. Interest in school size as an explanatory factor waxes and 
wanes, but never dies. The effect of variability in school size on 
educational achievement and a variety of related outcomes remains a 
subject of sometimes intense, sometimes dilatory, inquiry and debate. 

In the study reported here, we use a Texas data set representing 
1,001 high schools to build on previous research, completed first in 
California and then replicated in six very different states. (The data set 
is available here for researchers who wish to replicate or extend our 
analyses.) This line of research has, with unusual consistency, found 
ail interesting interaction effect between socioeconomic status (SES) 
and school size in the production of achievement: as school size 
increases, school performance (aggregate achievement at the school 
level) decreases for economically disadvantaged students. In short, as 
schools get larger, those with poor children as students perform 
increasingly less well when achievement is the outcome measure. 
School size imposes increasing "achievement costs" in schools serving 
impoverished communities. 

Research Questions 
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Continuing this line of research, we address two specific 
questions. First, will a replication that deploys a more fully specified 
regression model find the same size-by-SES interaction effect among 
the high schools in our Texas data set as was previously found (Bickel, 
1999b)? Second, whatever the merits of small schools, are large high 
schools with conventionally narrow grade ranges necessary to 
minimize expenditures per pupil ("save money"), or can "savings" 
occur without increased size? 

Replication Through Re-Specification 

In previous analyses, the independent variables included in 
regression equations were limited to a measure of school size, either 
total number of students or number of students per grade level; a 
measure of SES, most often percent of students eligible for free or 
reduced cost lunch; and the multiplicative interaction term. Some 
analyses included student-teacher ratio (Howley, 1999a, 1999b) or 
ethnicity variables (Bickel, 1999b). The most notable exception, 
however, is a multi-level analysis of Georgia data, which incorporated 
both ethnic composition and student/teacher ratio (Bickel & Howley, 
2000). To improve on past research, the primary difference between 
the work reported in the present study and the previous replications is 
a more fully specified regression model. 

Therefore, we are now asking if the size-by-SES interaction 
effect will prove unduly sensitive to better-informed regression model 
specification, diminishing the credibility of the consistent results 
reported from previous research. In other words, does the interaction 
effect merely mask the influence of "the usual suspects" through 
inadequate model specification? (Note 1) 

Fiscal Practicality 

In addition, we examine the claim that large schools with a 
narrow range of grades are a necessary organizational consequence of 
the modern need to minimize expenditures (fiscal efficiency). Many 
policy makers and administrators who have persisted in off-handedly 
dismissing the small-is-better research have done so in the name of 
fiscal practicality. Large consolidated schools, specializing in just a 
few grade levels, are viewed as essential to achieve "economies of 
scale" and to meet the supposedly critical developmental needs of 
students of differing ages. Those who hold contrary views are 
dismissed as romantics. (For a more balanced view, however, see 
Boex & Martinez- Vasquez, 1998). 

School size is negatively related to expenditure per pupil in 
zero-order correlational analysis. However, our analyses of the link 
between school size and expenditure per pupil go beyond the usual 
simplicities to include the under-researched concept of grade span 
configuration. (Note 2) 

Specifically, 1 16 of the high schools in our Texas data set are 
single-unit schools: the only school in a typically small, typically rural 
district, containing all elementary and secondary grades under a single 
roof (Note 3) With expenditure per pupil as the outcome measure, 
multiple regression analysis shows that single-unit schools, on 
average, correspond to a reduction in expenditure of $1 ,017 per pupil, 
a substantial efficiency, when compared with conventionally grade- 
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specialized high schools. (See Table 6.) 

The "savings" can be statistically attributed to two distinctive 
characteristics of single-unit schools in Texas: each is the only school 
in its district, and each has an unusually broad grade configuration, K- 
12, PreK-12, or early childhood-12 (see Table 7). We find, however, 
that the savings decline as such schools become larger. In other 
words, in Texas, small K-12 unit schools are cost effective, all else 
equal. They are also, as we shall see, educationally effective because, 
overall, such schools do tend to be small. 

School Size: A Timely Issue 

Writing on the role of school size as a determinant of school 
performance has a long history and is embedded in a voluminous 
literature (see, for example. Barker & Gump, 1964; Fowler, 1991; 
Guthrie, 1979; Khattari, Riley, & Kane, 1997; McDill, Natriello, & 
Pallas, 1986; Smith & DeYoung, 1988; Walberg & Walberg, 1994). 

As with so many commonly invoked explanatory factors in the social 
and behavioral sciences, reports about the effects of school size have 
been contradictory (Caldas, 1993; Lamdin, 1995; Rivkin, Hanushek, & 
Kain, 1998; Rossmiller, 1987). Part of the problem is that findings 
about size have often been a footnote in research focused on "effective 
schools," "school restructuring," or other species of broad-based 
reform efforts. As a consequence, school size sometimes has been 
relegated to the status of an obligatory but uninteresting control 
variable. Not infrequently, it simply has been ignored (Barr & 

Dreeben, 1983; Gamoran & Dreeben, 1986; Farkas, 1996; Hanushek, 
1997, 1998; Wyatt, 1996). 

Uncertainty as to the import of school size has yielded state-of- 
the-art school effectiveness research that fails to designate size a 
"resource," much less a resource worthy of investigation. A recent 
school effectiveness review by eleven production fonction virtuosos, 
for example, devoted four of its three hundred ninety-six pages to 
school size (Hodges & Greenwald, 1996, p. 81; Betts, 1996, pp. 166- 
168). Consequences of variability in school size were, in sum, judged 
to be uncertain. 

This assessment is simplistic and wrong according to recent 
studies. In fact, the Education Commission of the State (ECS) has for 
some time recommend smaller school size as one of the "best 
investments" policy makers could sponsor (Fulton, 1996). The 
research base on the influence of size per se (rather than as a feature of 
reformed or restructured schools) is developing quite rapidly, and may 
be said to have spawned a "movement" (Fine & Somerville, 2000). 

One Size Fits All 

One important limitation of most literature covering school size 
has been failure to examine the interaction of school size with other 
variables (Howley, 1989; Lee & Smith, 1995; Mik& Flynn, 1996; 
Riordan, 1997). This deficiency tends to give rise to a one-size-fits-all 
point of view. Within any school, it may seem, size-related benefits 
accrue and size-related costs are borne equally by all students (Conant, 
1959; Haller, 1992; Haller, Monk, & Tien, 1993; Hemmings, 1996). 
This turns out to be a dubious assumption (Bickel & Howley, 2000). 

Discounting Equity 
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In an era of cult-of-efficiency institutional restructuring, 
moreover, questions as to the "best" size for any school are often 
expressed in the scientific management terms of organizational 
efficiency. In economists' terminology, presumed economies of scale 
frequently have been given pride of place (Haller, Monk, Bear, 

Griffith, & Moss, 1990; Purdy, 1997; Tholkes & Sederberg, 1990). As 
with much contemporary educational research, equity questions are 
usually dismissed as irrelevant to the school size discussion, at least 
when fiscal efficiency is at stake. For many, this has simply come to 
mean that bigger is better, inevitably and always (Stevenson, 1996), 
when choices about school construction are made. 

Small is Better? 

Recently, nevertheless, attention has been drawn to a growing 
body of empirical research that holds that school size is negatively 
associated with conventional measures of educational productivity. 

This includes measured achievement levels, dropout rates, grade 
retention rates, and college enrollment rates (see, for example, Bickel 
& McDonough, 1997; Fowler, 1995; Fulton, 1996; Mik & Flynn, 

1996; Stevens & Peltier, 1995; Walberg & Walberg, 1994). 

Size-by-SES Interaction Effects 

In part, renewed interest in smaller schools is due to research 
concerning the joint or interactive, rather than independent or main, 
effects of school size and SES. Specifically, interaction effects have 
been identified which suggest that the well known adverse 
consequences of socioeconomic disadvantage are tied to school size in 
substantively important ways. 

In brief, as school size increases, the mean measured 
achievement of schools with less-advantaged students declines. The 
larger the number of less-advantaged students attending a school, the 
greater the decline (Bickel & Howley, 2000; Friedkin & Necochea, 

1988; Howley, 1995, 1996; Howley & Bickel, 1999; Huang & 

Howley, 1993). 

In addition to helping revive interest in school size as a variable 
of importance in educational research, this work has begun to sensitize 
researchers and policy makers to equity concerns associated with 
school size. One-size-fits-all is no longer a unanimous judgment. 

Some researchers and policy makers are now asking, "Best-size-for- 
whom?" (Devine, 1996; Henderson & Raywid, 1994). 

Reproducible Findings: A Research Agenda 

Research on size-by-SES interactions, moreover, has substeuitial geographic 
scope. The same school-level interactions have been found in California (Friedkin & 
Necochea, 1988), West Virginia (Howley, 1995, 1996), Alaska (Huang & Howley, 
1993); Montana (Howley, 1999a), Ohio (Howley, 1999b); Georgia (Bickel, 1999a; 
Bickel & Howley, 2000), and Texas (Bickel, 1999b). In contrast to so much research 
which has yielded initially interesting findings, the likelihood that additional 
replications will yield sharply conflicting results has been substantially reduced by 
the findings from these studies. 

Texas High School Data for 1996-97 
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By way of continuing this line of investigation, we use a data set consisting of 
1,001 Texas high schools. This represents 83.6 percent of all high schools in the 
state for academic year 1996-97. The 196 excluded schools are those for which 
values were not available for one or more of the variables used in our analyses 
(Bickel, 1999b). 

Independent Variables 

As already explained, previous research on this issue has been marked by 
simplified regression model specification. In large part, this parsimonious approach 
derives from the fact that proper specification for research on school size or any 
other correlate of achievement is substantively uncertain and theoretically thin. The 
usual suspects are SES and ethnicity variables, but a host of other variables has often 
been included in production function models. The debate continues (cf. Greenwald, 
Hedges, & Laine, 1996; Hanushek, 1996; Hedges, 1996). 

Pending a resolution to this debate, the independent variables included in 
Table 1 seem appropriate. (Note 4) They reflect the ethnic, linguistic, and 
socioeconomic diversity of the state's high school students (PCTBLACK, PCTHISP, 
PCTLEP, PCTPOOR); they show substantial variability in Texas high schools' 
organizational characteristics and resources, including size (SIZE, S/TRATIO, EPP, 
PCTINST, UNIT, LEVELS, HIGHSKLS); and they manifest pertinent variability in 
curricular composition (PCTTECH, PCTSPECL, PCTGIFT). (Note 5) 

Inclusion of student/teacher ratio (S/TRATIO), a useful proxy for class size 
among the additional independent variables, enables us to address questions as to 
whether small classes in large schools diminish the adverse consequences of 
increased size. As it turns out, they do not. This result is consistent with tests of the 
hypothesis in ancillary analyses provided in two of the previous studies (Howley, 
1999a, 1999b). 



Table 1 

Definitions of Variables 



SIZE 


Number of students. (Expressed in 
thousand- student units in Tables 3 through 
5; expressed in natural logarithms of single- 
student units in Tables 6 and 7.) 


PCTPOOR 


Percentage of students eligible for free or 
reduced-cost lunch. 


PCTBLACK 


Percentage of students who are Black. 
(Expressed in natural logarithms.) 


PCTHISP 


Percentage of students who are Hispanic. 


PCTLEP 


Percentage of students classified as limited 
English proficient. (Expressed in natural 
logarithms.) 


S/TRATIO 


Student/teacher ratio. 


EPP 


Expenditure per pupil. (Expressed in 
thousand- dollar units in Tables 3 through 

5.) 
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PCTINST 


Percentage of total budget allotted for 
instruction. 


PCTTECH 


Percentage of students enrolled in a full- 
time career and technical education 
curriculum. 


PCTSPECL 


Percentage of students enrolled in a full- 
time special education program. 


PCTGIFT 


Percentage of students classified as gifted. 


UNIT 


Coded 1 for single-unit schools, and 0 
otherwise. 


HIGHSKLS 


Number of high schools in a district. A high 
school is any school which includes grade 
12. (Expressed in natural logarithms.) 


LEVELS 


Number of grade levels. 


RIO 


Texas Assessment of Academic Skills tenth 
grade reading test. 


MIO 


Texas Assessment of Academic Skills tenth 
grade math test. 


WIO 


Texas Assessment of Academic Skills tenth 
grade writing rest. 



Dependent Variables: Measures of Achievement 

In Tables 3, 4, and 5, the dependent variables are taken from the mandatory 
Texas Assessment of Academic Skills (TAAS) end-of-grade battery, used on a 
limited basis since the Fall of 1990, and fully implemented in 1994. The tests are 
criterion-referenced measures of attainment in reading, math, and writing, 
administered to tenth graders throughout the state, and used to evaluate the 
performance of students and, by implication, the effectiveness of schools and school 
districts in promoting measured achievement. Measures of internal consistency for 
the TAAS are reported to range from .80 to .90 (Texas Education Agency, 2000). 
(For critical discussions of the use and interpretation of TAAS, see Clopton, Bishop, 
& Klein ,1997; Haney, 2000; and Klein, Hamilton, McMaffery, & Stecher, 2000). 

Dependent Variables: Expenditure Per Pupil 

In Tables 6 and 7, expenditure per pupil is the dependent variable, and 
measured achievement is used for purposes of statistical control rather than as an 
outcome measure. Since scores for RIO, MIO, and WIO are closely correlated, use 
of all three in the same equation produces multicollinearity, with Condition Indices 
well above thirty (Gujurati, 1995, p. 338). 

To eliminate this threat, we have created a summary achievement measure, 
COMPOSITE, which is the sum of the Z scores of RIO, MIO, and WIO. All 
bivariate correlations between COMPOSITE and its three constituents exceed .935. 

We have also found that the relationship between SIZE and EPP is curvilinear, 
but that the relationship can be linearized using natural logarithms of SIZE. Use of 
this transformation is discussed further in the next section. 

Descriptive Statistics 

o 
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Table 2 shows that the mean value for SIZE, total number of students 
enrolled, is 877.19. The size of the standard deviation, 849.88, indicates that SIZE 
manifests a good deal of variability, with a coefficient of variation of 103.2 percent. 

While SIZE has a positive skew, the skew is not so extreme that the variable 
warrants logarithmic or other transformation (Fox, 1997, pp. 64-68). In fact, using 
the Studentized range test for normality, SIZE more closely approximates a normal 
distribution when non-transformed values are used (see Kanji, 1993, p. 65). 
Therefore, actual SIZE values are used in the analyses with achievement tests as 
outcome measures, reported in Tables 3, 4, and 5. 

The relationship between SIZE and EPP is curvilinear: concave and sloping 
downward for the smallest values of school size; almost perfectly straight with a 
modest downward slope for SIZE values between 220 and 550; almost perfectly 
straight with a diminished downward slope between size values 550 and 1800; then 
sloping still less, and eventually becoming level for SIZE values of more than 3200 
students. This is similar to the curvilinear relationship between high school size and 
cost discovered by Stiefel, Berne, latarola, & Fruchter (2000) in their New York 
City data. 

We have linearized the relationship between SIZE and EPP in our Texas data 
by taking natural logarithms of SIZE for the analyses reported in Tables 6 and 7 
(where EPP is the dependent variable).. 

Table 2 Descriptive Statistics 
Means and 

(Standard Deviations) 

N=1001 



SIZE 


877.19 

(849.88) 


PCTPOOR 


36.51 

(30.93) 


PCTBLACK 


11.07 

(17.34) 


PCTHISP 


27.73 

(27.78) 


PCTLEP 


4.95 

(8.99) 


S/TRATIO 


13.24 

(3.15) 


EPP 


4745.67 

(1318.94) 


PCTINST 


69.92 

(7.34) 


PCTTECH 


56.12 

(20.59) 


PCTSPECL 


13.54 

(6.08) 


PCTGIFT 


9.02 

(7.07) 




0.12 
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(0.32) 


HIGHSKLS 


2.96 

(5.12) 


LEVELS 


5.34 

(3.11) 


RIO 


39.17 

(2.30) 


MIO 


45.51 

(4.08) 


WIO 


32.88 

(1.80) 



Means and standard deviations for PCTBLACK, PCTLEP and HIGHSKLS 
are reported in Table 2 before the variables were logged. Since, however, each has a 
sharp positive skew, with most of the observations confined to a very narrow range 
of data on the left side of the distribution, the variability of each is tightly 
constrained. Taking natural logarithms spreads each distribution, making it more 
informative (Fox, 1997, pp. 64-68). 

It is also worth noting that the standard deviations for the RIO, MIO, and WIO 
achievement tests are small: 2.30, 4.08, 1.80. Coefficients of variation are similarly 
small, 5.9 percent, 9.0 percent, and 5.5 percent. 

Routine tests for violations of assumptions of the classical normal linear 
regression model, and for the presence of influential observations ("outliers"), were 
conducted. No assumptions were violated, and there were no speciously influential 
observations. 

Regression Results: A Robust Interaction Effect 

Tables 3, 4, and 5 provide results of regression analyses using TAAS reading, 
math, and writing scores as dependent variables. The most interesting finding for 
present purposes is that the size-by-SES interaction effect is statistically significant 
and negative in each instance. This interaction, in each case, is, in fact, the most 
influential variable after SES and the ethnicity variables (the influence of which 
varies across subject areas). As school size increases, the cost to school performance 
of schools serving economically less-advantaged students increases, as well. This, of 
course, was the finding in all previous replications. 

Table 3 

TAAS Reading Achievement 
Unstandardized and (Standardized) Coefficients 

N=1001 



SIZE 


0.177 (.065) 


PCTPOOR 


-0.040*** (-.367) 


PCTBLACK! 


-0.253*** (-.142) 


PCTHISP 


-0.010** (-.123) 


PCTLEP! 


-0.268** (-.117) 


S/TRATIO 


-0.008 (-.011) 


EPP 


0.027 (.015) 



II ii 



httn7/enaa a.sii.eHii/p:nafi/vQn40 html 



333 



10/30/01 





EPAA Vol. 9 No. 4...: Grade Span Configuration, Achievement, and Expenditure per Pupi Page 10 of 32 



PCTINST 


0.007 (.022) 


UNIT 


0.733** (.102) 


PCTTECH 


0.004 (.040) 


PCTSPECL 


0.047** (-.123) 


PCTGIFT 


0.038** (.118) 


SIZE-by-SES 


-0.035** (-.143) 


Adjusted R-Squared = 40.3% 
*** <.001 
** <.01 
*<.05 

! Expressed as Natural Logarithms 


Partial Derivative = • 


-0.035(PCTPOOR) 


Effect Size 
Points (S.D. Units) 


PCTPOOR 

(Quartiles) 


-0.76 (-0.33) 


21.6% 


-1.14(-0.50) 


32.5% 


-1.73 (-0.75) 


49.5% 


-3.50 (-1.52) 


100.0% 



Table 4 

TAAS Math Achievement 
Unstandardized and (Standardized) Coefflcients 

N=1001 



SIZE 


0.019 (.040) 


PCTPOOR 


-0.062*** (-.318) 


PCTBLACK! 


-0.631*** (-.200) 


PCTHISP 


-0.022** (-.152) 


PCTLEP! 


0.010 (.002) 


S/TRATIO 


-0.146** (-.113) 


EPP 


-0.149 (-.048) 


PCTINST 


0.007 (.013) 


UNIT 


0.611 (.048) 


PCTTECH 


0.005 (.024) 


PCTSPECL 


-0.064** (-.095) 


PCTGIFT 


0.052** (.090) 


SIZE-by-SES 


-0.060** (-.144) 


Adjusted R-Squared = 30.5% 


*** <.001 
** <.01 
* <.05 

! Expressed as Natural Logarithms. 
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Partial Derivative = -0.060(PCTPOOR) 


Effect Size 
Points (S.D. Units) 


PCTP9OR 

(Quartiles) 


-1.30 (-0.32) 


21.6% 


-1.95 (-0.48) 


32.5% 


-2.97 (-0.73) 


49.5% 


-6.00 (-1.47) 


100.0% 



Table 5 

TAAS Writing Achievement 
Unstandardized and (Standardized) Coefficients 

N=1001 



SIZE 


0.052 (.025) 


PCTPOOR 


-0.031*** (-.366) 


PCTBLACK! 


-0.183*** (-.132) 


PCTHISP 


-0.002 (-.037) 


PCTLEP! 


-0.310*** (-.173) 


S/TRATIO 


-0.041 (-.072) 


EPP 


-0.007 (-.006) 


PCTINST 


0.007 (.027) 


UNIT 


0.505** (.090) 


PCTTECH 


-0.001 (-.010) 


PCTSPECL 


-0.036*** (-.123) 


PCTGIFT 


0.027** (.105) 


SIZE-by-SES 


-0.033*** (-.171) 


Adjusted R-Squared = 40.3% 


*** <.001 
** <.01 
*<.05 

! Expressed as Natural Logarithms. 


Partial Derivative = • 


-0.033(PCTPOOR) 


Effect Size 
Points (S.D. Units) 


PCTPOOR 

(Quartiles) 


-0.7U-0.40) 


21.6% 


-1.07 (-0.60) 


32.5% 


-1.63 (-0.91) 


49.5% 


-3.30 (-1.84) 


100.0% 



Clearly, the interaction effect involving school size and the percentage of 
students who are poor is robust and strong in the presence of regression model re- 
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specification. This result adds credibility to the repeatedly replicated finding that 
smaller schools diminish the achievement disadvantages associated with being poor. 
Larger schools, by contrast, exaggerate these disadvantages. 

Effect Size 

As with previous research on size-by-SES interactions, we have computed 
illustrative effect sizes by using partial derivatives. This is done by differentiating 
the regression equations in Tables 3 through 5 with respect to SIZE (expressed in 
thousand-student units), while treating the other independent variables as constants 
(Purcell and Varberg, 1984, pp. 308-309, 636-639). Statistically nonsignificant 
coefficients are set equal to zero. (Note 6) 

The results, reported at the bottom of each table, are the average achievement 
decrements, in test score points and standard deviation (S.D.) units, which come 
with each quartile increment in PCTPOOR. In each instance, we see that there are 
mean achievement test score costs associated with economically disadvantaged 
students, and these costs increase as the percentage of less-advantaged students 
increases. 

The substantial nature of the achievement costs becomes clearer when we 
recall that the standard deviations and coefficients of variation for RIO, MIO, and 
WIO are small. This replication, based on informed regression model re- 
specification, makes clear that the size-by-SES interactions are robust and strong. 

Can Costs Decline Without Increasing Size? 

In spite of the consistently strong findings about school performance, small 
schools with a broad range of grade levels seem to many— if not most— observers 
singularly anachronistic. The move toward ever-larger, ever-more grade-specialized 
schools, is proceeding apace (Lyons, 1999; Funk & Bailey, 1999; Boex & Martinez- 
Vasquez, 1998). One of the coauthors recently received a query from a former 
student about whether any research addressed the greater effectiveness of a K-3 
versus aK-5 school. The answer, not surprisingly, is "no." 

This study, however, together with several other studies (e.g., Franklin & 
Glascock, 1998; Howley & Harmon, 2000a; Wihry, Coladarci, & Meadow, 1992; 
DeYoung, Howley, & Theobald, 1995), attempts to raise the issue of grade span 
configuration more systematically . In general, the present analysis finds that 
restricting the grade span of a school increases costs. That is, given a level of school 
performance, the school with a broader grade span will provide that level of 
performance at lower cost (all else equal). 

A critical problem for such an analysis is differences in grade level 
expenditures per pupil, which are higher for secondary than for elementary grades. 
Without controlling for this difference, we bias the analysis to favor cost reductions 
for schools with the broadest range of grades. Therefore, we created a weighting 
variable for the EPP at each grade level to control for such grade level differences in 
expenditure per pupil. This additional variable (which does not appear in the tables) 
was created by multiplying the number of students at each grade level by the mean 
EPP at each grade level, summing across the grades included in a school, then 
dividing by school size. (Note 7) 

We make these analyses because, administrators and policy makers deal with 
fiscal constraints that render findings about the educational benefits of small size 
seem impractical to them. For them, cost remains a primary consideration. For 
instance, rural superintendents who operated small rural high schools (enrolling 
fewer than 400 students) recently cited fiscal constraints as the primary threat to the 
continued existence of such schools (Howley & Harmon, 2000b). Departure from 
the large, grade-specialized mode in pursuit of equity appears to many 
administrators to be a luxury they cannot afford (Keller, 2000). Findings reported 
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here should help administrators and policy makers revise commonly held views 
about the fiscal practicality of operating small high schools in the 7-12 and K-12 
configurations. (Note 8) 

Multiple Regression Analysis: Expenditure Per Pupil 

In the regression analysis reported in Table 6, the dependent variable is 
expenditure per pupil. The independent variables are otherwise the same as with 
Tables 3, 4, and 5, except that the size-by-SES interaction term has been deleted as 
irrelevant to this analysis (since the theory links the interaction to school 
performance, which is not the dependent variable in these analyses), and the three 
achievement test scores are now used as independent variables for purposes of 
statistical control, appearing jointly in the COMPOSITE variable. 

Finally, a multiplicative interaction term created using UNIT and SIZE (with 
SIZE logged and centered, see Cronbach, 1 987) has been added. Given statistically 
significant coefficients for these two variables, a UNIT-by-SIZE interaction term 
will enable us to determine if the relationship between SIZE and EPP varies between 
single-unit schools and conventional high schools. 

Table 6 

Unit Schools and Expenditure Per Pupil 
Unstandardized and (Standardized) Coefficients 

N=1001 



SIZE! 


-254.415*** (-.199) 


PCTPOOR 


-4.158 (-.066) 


PCTBLACK! 


81.239** (.080) 


PCTHISP 


5.668** (.119) 


PCTLEP! 


37.920 (.029) 


S/TRATIO 


-284.614*** (-.680) 


PCTINST 


-35.422*** (-.199) 


PCTTECH 


-2.923 (-.046) 


PCTSPECL 


1.291 (.006) 


PCTGIFT 


4.823 (.026) 


COMPOSITE 


-3.551 (-.008) 


UNIT 


-1017.607*** (-.247) 


UNIT-by-SIZE 


-730.195*** (-.172) 


Adjusted R-Squared = 51.4% 


*** <.001 
** <.01 
*<.05 

! Expressed as Natural Logarithms. 

1 1 Weighted for differences in mean EPP by grade 
level. 


Partial Derivative = -254.415(1/SIZE) - 730.195 
(UNIT)(1/SIZE) 


Effect Size 


SIZE 
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(Dollars) 


(Quartiles) 


UNIT=1 UNIT=0 




-4.48 -1.16 


220 


-2.20 -0.57 


447 


-0.67 -0.17 


1459 


-0.22 -0.06 


4434 



Regression Results: Anticipated and Unanticipated Findings 

Not surprisingly, school size (SIZE) has a statistically significant and negative 
relationship to expenditure per pupil. The same is true of student-teacher ratio 
(S/TRATIO), the variable exercising the greatest influence on expenditure per pupil. 
Smaller schools and smaller classes are associated with higher expenditures overall 
(but not with all else equal). 

Less predictably, the statistically significant regression coefficient 
corresponding to UNIT is notable: Being a single-unit school is associated with an 
average reduction in expenditure-per-pupil of just over $1,017. Other things being 
equal (that is, with the fiill complement of controls in place, including achievement 
level, class size, and grade-level differences in EPP), having only one school, 
covering all grades in a district, represents substantial dollar savings. 

The multiplicative interaction term, UNIT-by-SIZE, however, also has a 
negative and statistically significant coefficient. This interaction indicates that the 
net influence of increases in school size provides more substantial cost reductions 
for single-unit schools than for conventional schools. 

Reduced Costs Without Increased Size? 

The results reported in Table 6 affirm the conventional wisdom that size is 
negatively related to expenditure per pupil, for both single-unit schools and 
conventional high schools. Table 6 also shows, however, that the relationship is 
more complex than commonly acknowledged. After controlling for size and a 
reasonable complement of other factors, single-unit schools are associated with 
substantial savings in expenditure per pupil, and increases in size yield greater cost 
reductions for single-unit schools than for conventional grade-specialized schools. 
What can explain such unexpected findings? We seek possible answers to such 
questions in the organizational distinctiveness of single-unit schools as defined in 
this study. 

Single-Unit Schools: Organizational Distinctiveness 

Organizationally, the characteristics that conspicuously set these single-unit 
schools apart are number of grade levels, and the fact that, in this data set, each is the 
only school in its district. (Note 9) Seventy-five percent of the high schools in our 
data set have four or fewer grades (LEVELS). Single-unit schools, however, with K- 
12, PreK-12, or early childhood- 12 configurations, have thirteen, fourteen, or fifteen 
grade levels. Similarly, the mean of the variable HIGHSKLS (before logging) tells 
us that the average number of high schools per district is nearly three, while a single- 
unit school is the only school of any kind in its district. 

Single-Unit Distinctiveness and Expenditure Per Pupil 

In an effort to explain cost savings associated with single-unit schools. 
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therefore, in Table 7 we have added two additional independent variables, 
representing the distinctive characteristics of single-unit schools. Since LEVELS is 
very closely correlated with UNIT (r=.965), the UNIT variable has been deleted, 
replaced by the organi 2 ational components of the Texas single-unit school 
phenomenon (i.e., LEVELS and HIGHSKLS). We construe the new independent 
variables as essential components of the global, complex variable UNIT (Rosenberg, 
1968, pp. 40-52). In effect, we are trying to identify the specific characteristics of 
UNIT that may account for its unexpected relationship with expenditure-per-pupil 
(EPP). These characteristics, of course, may also be associated with reduced costs in 
conventional high schools. 

We have also created a multiplicative interaction term with SIZE and each of 
the components of UNIT. Thus, we are also adding to the regression equation 
LEVELS-by-SIZE and HIGHSKLS-by-SIZE, with all variables used in creating the 
interaction terras centered (Cronbach, 1 987). 

Table 7 

One High School, Grade Levels, and Expenditure Per Pupil 
Unstandardized and (Standardized) Coefficients 

N=1001 



SIZE! 


-290.519*** (-.227) 


PCTPOOR 


-2.927 (-.046) 


PCTBLACK 


35.476 (.035) 


PCTHISP 


4.160* (.088) 


PCTLEP! 


23.216 (.018) 


S/TRATIO 


-314.462*** (-.751) 


PCTINST 


-34.101*** (-.191) 


PCTTECH 


-3.365 (-.053) 


PCTSPECL 


1.318 (.006) 


PCTGIFT 


0.646 (.003) 


COMPOSITE 


8.725 (.019) 


HIGHSKLS 


332.023*** (.223) 


LEVELS 


-98.358** (-.232) 


HIGHSKLS-by-SIZE 


-114.038* (-.076) 


LEVELS-by-SIZE 


-48.445** (-.108) 


Adjusted R-Squared = 52.8% 


*** <.001 
** <.01 
* <.05 

I Expressed as Natural Logarithms. 

! ! Weighted for differences in mean EPP by grade 
level. 


Partial Derivative = - 290.519(1/SIZE) - 1 14.038 
(HIGHSKLS)(1/SIZE) - 48.445(LEVELS)(1/SIZE) 
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Effect 

Size 

(Dollars) 


SIZE 

(Quartiles) 


HIGHSKLS 

(Quartiles) 


LEVELS 

(Quartiles) 


-2.20 


220 


0 


4 


-1.08 


447 


0 


4 


-0.41 


1459 


0.69 


6 


-0.31 


4434 


3.26 


15 



LEVELS, HIGHSKLS, and Expenditure Per Pupil 

The results are instructive. Predictably, as with Table 6, the coefficients 
corresponding to SIZE and S/TRATIO are negative and statistically significant. This 
holds in spite of the fact that SIZE and S/TRATIO are substantially correlated 
(r=.736), thereby reducing statistical power. However, the variance inflation factors 
for each, though the largest for the equation, are well within acceptable limits, 4.870 
and 4.131 (Chatterjee, Hadi, & Price, 2000, pp. 240-241). (Note 10) 

Furthermore, given that LEVELS and HIGHSKLS are construed as effective 
components of UNIT, the following results are not surprising; as the number of high 
schools in a district increases, expenditure per pupil also increases, averaging just 
over 332 dollars per school. In addition, each grade level added to a high school is 
associated with an average expenditure per pupil decrease of just over 98 dollars. 
(The distribution of high schools per district and by grade levels is reported in Table 
8 and Table 9.) 



Table 8 

High Schools Per District 



1 


2 


3 


4 


5 


6 


8 


10 


12 


21 


26 


92.49% 

(727) 


3.69% 

(29) 


1.14% 

(9) 


1.27% 

(10) 


0.76% 

(6) 


0.51% 

(4) 


0.25% 

(2) 


0.25% 

(2) 


0.13% 

(1) 


0.13% 

(1) 


0.13% 

(1) 



Table 9 

Grade Levels Per High School 



2 


3 


4 


5 


6 


7 


9 


12 


13 


14 




0.50% 

(5‘) 


1.80% 

(18) 


72.43% 

(725) 


0.80% 

(8) 


11.09% 

(111) 


1.70% 

(17) 


0.09% 

(1) 


0.09% 

(1) 


5.79% 

(58) 


3.70% 

(37) 


2.00% 

(20) 



Finally, the statistically significant interaction terms make clear that as SIZE 
increases, the increased costs associated with having more than one high school in a 
district are diminished; while the reduced costs associated with having more grade 
levels are reduced still more. 

What Is To Be Made of All This? 

School Size and Expenditure Per Pupil: Diminishing Returns 

One way to summarize these complex results is to refer to the illustrative 
effect sizes reported (by quartiles of significant variables) on Tables 6 and 7. For 
each analysis, as school size increases, the partial derivatives show savings, but 



httn7/e,naa a<5ii pdn/p.naa/vQn4n html 



340 



10/30/01 


























EPAA Vol. 9 No. 4...: Grade Span Configuration, Achievement, and Expenditure per Pupi Page 17 of 32 



progressively diminished savings. (Note 1 1) School size is negatively related to 
expenditure per pupil, but savings diminish with each increment in size (see the 
following discussion of diseconomies of scale for our interpretation of this finding.) 

School Size and Expenditure Per Pupil: Single-Unit Schools 

Furthermore, with a judiciously selected complement of controls in place, 
single-unit schools and their defining characteristics— number of grade levels and 
uniqueness in their district— are associated with substantial savings in expenditure 
per pupil. For these organizationally distinctive schools, moreover, size contributes 
more to reducing costs than in conventional high schools. 

One related observation needs still to be underscored. Despite the comparative 
cost-advantages of increased size for single-unit schools, change in the rate of 
reduction in EPP as size increases slows for both K-12 schools and other schools— 
the slowing is simply less dramatic for other schools. See the effect sizes given in 
Tables 6 and 7 to gauge this difference. (Note 12) 

School Size and Expenditure Per Pupil: HIGHSKLS and LEVELS 

Not surprisingly, given the savings associated with single-unit schools, as the 
number of schools in a district increases, so does expenditure per pupil, though this 
additional cost is less for larger schools than for smaller schools. While this finding 
might suggest that building additional large, as compared to small, schools is cost- 
effective, readers need to recall two other facts. First, the law of diminishing returns 
to investment is definitely applicable: Ever-larger size assures ever-diminishing 
returns with regard to expenditure per pupil. Second, larger consolidated schools 
typically do have conventionally narrow grade spans, and, as the number of grade 
levels in a school decreases, expenditure per pupil is again increased. So far as 
expenditure-per-pupil goes, size (total enrollment), grade span configuration, and 
district organization structure a quite complex playing field for the game of 
minimizing costs. "Larger schools cost less to operate" is not even a close 
approximation of such complexity. 

Most succinctly: Bigger is not always or even usually cheaper. The questions 
to be answered locally are: (1) how big (when do the returns to increased size yield 
negligible savings)? (2) bigger for whom (poverty, ethnicity— poorer communities 
require smaller schools to maximize achievement)? and (3) bigger under what 
circumstances (district organization and grade span configuration)? The analyses 
presented so far show that answers to questions 2 and 3 constrain the answer to 
question 1 . Those who govern school fonding and school construction have not, to 
our knowledge, even begun to recognize the real constraints to large size. 

Diseconomies of Scale 

Typically, economists attribute diseconomies of scale to problems posed by 
the need for coordination and control (Bidwell & Kasarda, 1975; Boex & Martinez- 
Vasquez, 1998; Friedman, 1990). This observation follows from different interests 
among organizational participants, including lack of consensus with regard to 
organizational objectives. The usual response is a system of personnel and 
procedures for supervision and monitoring: bureaucratic organization. Supervision 
and monitoring are costly additions to an organization, but in increasingly large 
organizations these additional costs are (ironically) increased by the need to 
coordinate and control those who supervise and monitor. Bureaucratic organization, 
a feature of increased organizational scale, inevitably has the effect of complicating 
organization itself. This is a concern faced by any large organization, not just 
schools. 

As organizations become larger and more complicated, with ever-greater 
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specialization among employees, departments, and levels, threats of organizational 
anomie and anarchy not only come into play, but are often realized and 
disorganization begins to prevail (Shedd & Bachrach, 1991). In dynamic fashion, 
additional "negative feedback loops" necessary to maintain stability increase 
supervision and monitoring costs to unacceptably high— and ultimately 
counterproductive— levels. Change and adaptation become so costly that they are 
sacrificed to the imperative of sheer survival. A school enrolling 750 students can 
easily offer all the curricular and co-curricular "iconography" that characterizes the 
American comprehensive high school (Haller et al., 1990), and increases in size 
beyond some hypothetical level of what might be called "programatic surfeit" come 
at a cost to efficiency, recognizable as diminishing returns to size if not as absolute 
diseconomies of scale. For instance, high schools enrolling 3,500 as compared to 
750 students would (hypothetically) realize little or no economic advantage to their 
increased size, they might encounter diseconomies of scale that counterbalance and, 
beyond some hypothetical threshhold, overwhelm the accumulated advantages of 
economies of scale (see, e.g., Bidwell & Kasarda, 1975; Friedkin & Necochea, 
1988). 

This description will sound familiar to many readers in big-city mega-districts 
as well as to readers of very large districts in rural and suburban locales. The so- 
called "small schools movement" is a reform strategy to address this dilemma in 
metropolitan districts. Elsewhere, in rural areas and small towns, extant small high 
schools are often regarded as too expensive to exist, in part because analyses with 
adequate controls (such as appear in the present study) are so seldom undertaken or 
even understood as necessary. According to some observers, both policy analysts 
and policy makers have tended to ignore the issue of organizational scale as an 
influence on school performance (Guthrie, 1979; Howley, 2000; Wasley, Fine, 
Gladden, Holland, King, Mosak, & Powell, 2000) 

The results of this study may indicate that inclusion of all grade levels in the 
same setting fosters a common, perhaps strongly tacit, understanding of 
organizational purpose. A K-12 school, for example, includes all personnel who 
teach and administer in all grades in the same location. This may foil development 
of the usual articulation problems that characterize relationships among elementeiry 
schools, middle schools, and high schools, diminishing the need for costly 
monitoring and supervision. 

Similarly, if a school is the only one in its district, between-school differences 
in purpose and procedure cannot occur, further reducing the need for coordination 
and control through monitoring and supervision. When a single school with a broad 
range of grade levels is also small, the seemingly antithetical goals of saving money 
and promoting equity in achievement may well be attained simultaneously; the odds 
of doing so are at any rate increased, according to the analyses in this article. 

This tentative account, of course, shifts our focus from schools to school 
districts. This is consistent with earlier Georgia research, in which we found that the 
achievement of less-advantaged students in larger schools was diminished less if the 
schools were located in smaller districts. In addition, we found that the expected 
achievement gains of less-advantaged students in small schools were undercut in 
large districts (Bickel & Howley, 2000; Howley, 2000). 

Size-by-SES and Cost 

Table 10 joins the size-by-SES and cost issues still more closely together. We 
use the same regression model specification employed in Table 7. Our achievement 
composite is now the outcome measure, and we reintroduce the size-by-SES 
interaction term. 



Table 10 

Composite Achievement 
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Unstandardized and (Standardized) Coefficients 

N=1001 



SIZE! 


0.218 (.079) 


PCTPOOR 


-0.054*** (-.403) 


PCTBLACK! 


-0.270*** (-.123) 


PCTHISP 


-0.008 (-.081) 


PCTLEP! 


-0.255* (-.090) 


S/TRATIO 


0.017 (.019) 


PCTINST 


0.004 (.011) 


PCTTECH 


0.001 (.009) 


PCTSPECL 


-0.056*** (-.121) 


PCTGIFT 


0.051*** (.130) 


HIGHSKLS! 


-0.946*** (-.297) 


LEVELS 


0.130** (.142) 


HIGHSKLS-by-SIZE 


0.534*** (.166) 


LEVELS-by-SIZE 


0.050 (.051) 


SIZE-by-SES 


-0.034* (-.116) 


Adjusted R-Squared = 42.7% 

*** <.001 
*♦ <01 
*<.05 

! Expressed as Natural Logarithms. 


Partial Derivative = 0.534(HIGHSKLS)(1/SIZE) - 
0.034PCTPOOR) 


Effect Size 
Points (S.D. 
Units) 


SIZE 

(Quartiles) i 


HIGHSKLS 

[Quartiles) 


PCTPOOR 

(Quartiles) 


-0.73 (-0.26) 


220 1 


0 


21.6 


-1.10 (-0.39) 


447 1 


0 


32.5 


-1.68 (-0.59) 


1459 1 


0.69 


49.5 


-3.39 (-1.20) 


4434 


3.26 


100.0 



Interestingly, LEVELS, the component of UNIT which was associated with 
reduced expenditures, is now associated with increased achievement. HIGHSKLS, 
the component of UNIT which was associated with increased expenditures, is now 
associated with decreased achievement. 

In most other respects the results in Table 10 are like the results reported in 
Tables 3, 4, and 5. Once again, the size-by-SES interaction term is statistically 
significant and negative (equal in magnitude to PCTBLACK! and PCTSPECL, i.e., 
= -.116, p<.05), and the illustrative effect sizes demonstrate that as school size 
increases, the presence of economically disadvantaged students is associated with 
diminished average achievement. 

Most significantly, perhaps, the influence of size across the SES spectrum. 
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from relatively affluent to impoverished, is negative, though the influence of size is 
most harmful in larger districts serving many poor students (effect size = -1.20, see 
Table 10). Even in small unit schools serving a relatively affluent community, 
however, data in Table 10 show that a one standard-deviation-unit increase in size 
(850 students, see Table 2), would depress school performance by about one-fourth 
of a standard deviation. 

Cautions 

Our data set contains a large number of cases and a broad range 
of pertinent variables. Nevertheless, it is useful to bear in mind that 
Texas is a distinctive state. For this reason, our analysis is limited in 
specific ways (to be considered shortly). We do not claim that these 
results necessarily apply in other states; indeed, many states retain no 
single-unit schools, and the present analysis can not be completed in 
them. 

However, the Texas case also shares certain features of policy 
context with other states. First, most states continue to make changes 
to their accountability schemes, and these changes notably include 
changes to assessment instruments. Texas (and many other states) 
claim, for instance, to be creating "tougher" tests all the time. Such 
changes are usually more cosmetic than substantive, and there is little 
reason to suspect that even substantive changes would dramatically 
alter relationships that prevail among influences in the present study. 

The fact also remains that the previous studies in this line of research 
have analyzed data from different states and employed different sorts 
of achievement measures (both norm- and criterion-referenced 
standardized tests) with rather consistent results. We would predict 
that the realtionships apparent here would persist with marginally 
different sorts of tests— somewhat "tougher," "more authentic," or 
measuring incrementally different achievement constructs. 

Second, in Texas, as in other states, school finance litigation 
continually produces marginal changes in how schools are funded. 
Nonetheless, it remains an American principle that schools in wealthy 
communities sustain their funding advantages through all such 
changes. Political contest, after all, revolves around the way money is 
deployed by powerful interests for public and private purposes, and 
poor people are not well positioned to prevail in such contests. For 
instance, power equalization school finance schemes may, in Texas 
and elsewhere, mute the relationships reported here, but they are 
unlikely to substantially obscure them. Our use of tests of statistical 
significance serves as a modest hedge against the effect of incremental 
policy movement, such as changes in assessment and finance systems 
may entail. 

Model Specification 

Misleading results due to specification error are a good deal less 
threatening in our achievement analyses than in our analyses of 
expenditure per pupil. The size-by-SES interaction effect has proven 
robust across seven very different states, and for at least four different 
regression model specifications, two in this paper alone. (Compare 
Tables 3, 4, and 5 with Table 10. Also see Bickel & Howley, 2000; 

Friedkin & Necochea, 1988; Howley, 1995; Howley & Bickel, 1999; 

Huang & Howley, 1993). 

Misleading results due to specification error are more likely in 
our analyses of expenditure per pupil because the variables we have 
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found to be especially interesting, UNIT, LEVELS, and HIGHSKLS, 
as well as the interaction effects created with SIZE, have not been 
adequately researched. 

The research that has been done on these issues, moreover, does 
not address relationships between expenditure and variables such as 
UNIT, LEVELS, and HIGHSKLS (see Wihry, Coladarci, and 
Meadow, 1992; Alspaugh, 1996; Howley & Harmon, 2000a; Franklin 
& Glascock, 1998). Therefore, though our choice of independent 
variables and functional forms seems reasonable, our regression model 
specification is necessarily tentative, and we readily acknowledge that 
a better-informed alternative might yield different results. 

Concepts: Single-Unit School 

We have defined single-unit schools as the only school in a 
district, including all grade levels. The performance of the component 
variables LEVELS and HIGHSKLS, along with interaction effects 
created with these variables and SIZE, suggests that there is merit to 
this way of construing the single-unit school and its distinctive 
components. 

However, in the only national survey of single-unit schools, 
Howley & Harmon (2000a) suggest that the single-unit designation be 
applied to any K-12 school, whether or not it is the only school in its 
district. In Texas, however, each such school is, in fact, the only 
school in its district. In a real sense, as we have seen, Texas single-unit 
schools are districts as well as schools. 

This account of the simultaneous realization of the supposedly 
competing objectives of equity and cost efficiency suggests that 
having more than one single-unit school in a district would diminish 
its attractiveness. The uniqueness-in-district that is a defining 
characteristic of single-unit schools in Texas is a common feature of 
many K-12 schools (Howley & Harmon, 2000). But it does not 
characterize all single-unit schools still in existence. 

Concepts: Expenditure Per Pupil 

We have measured cost in terms of expenditure per pupil. Funk 
and Bailey (1999), however, in their Nebraska research, judged cost 
per graduate to be a superior measure of cost efficiency. After all, one 
virtue of smaller school size is a lower dropout rate. 

Similarly, Stiefel, Berne, latarola, and Fruchter (2000) measured 
cost in terms of total budget per pupil and total budget per graduate. 
Neither measure revealed the cost inefficiencies commonly attributed 
to small schools. 

Whatever the virtues of per-graduate measures, their calculation 
requires dropout data which covers all grades in the schools being 
analyzed (Stiefel, Berne, latrola, & Fruchter, 2000, p. 33). Twenty-five 
percent of our Texas high schools, however, have five or more grades, 
and information on dropouts is often not reported for lower grades. 

Our choice of the traditional expenditure per pupil measure, therefore, 
was dictated by the information available in our Texas data set. Its use, 
together with use of our grade-level-expense weighting variable 
(described previously), nonetheless means that the findings reported 
here probably represent conservative estimates of cost efficiency. 

Multi-Level Analysis? 
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With the individual high school as the unit of analysis, an 
obvious strategy would be to conduct a multi-level analysis, with 
school districts constituting the second level (schools within districts). 
As it turns out, however, while only 1 1 .6 percent of the schools are of 
the single-unit variety, 72.6 percent of the districts operate just one 
high school. This yields an average within-group sample size of 1.27. 
High schools and districts are thoroughly confounded in the 
organizational structure of public secondary education in Texas, a 
situation common to many states. In short, for this analysis, the multi- 
level approach is simply inapt. (Note 13) 

In addition. Singer (1987) has shown that with small within- 
group sample sizes, and small residual intra-class correlations, 
standard errors of regression coefficients are diminished very little by 
intra-class correlation, and tests of significance are reliable ^ote 14). 
In all our analyses, deflation of standard errors due to intra-class 
correlation is less than two percent (Singer, 1987, pp. 224-226). 

Conclusions 

As with seven previous analyses, we have found that as school 
size increases, achievement test score costs associated with having 
economically disadvantaged students in schools increase, as well. This 
finding has now proven robust across seven states and at least four 
different regression model specifications. This degree of consistency is 
rare, indeed, in educational research. 

We have also found that, while administrators and policy 
makers are correct in their judgment that school size is negatively 
related to costs, that is far from the whole story, at least with regard to 
expenditure per pupil. The negative relationship between size and 
expenditure per pupil becomes increasingly tenuous as school size 
increases, and eventually savings become negligible. 

In addition, organizational factors, especially as manifest in the 
distinctive components of the single-unit school, reveal unanticipated 
relationships to cost reduction. If we were designing schools solely to 
minimize expenditure per pupil (an educationally counterproductive 
goal in the view of the authors), the best configuration might very well 
be a large single-unit school. 

However, if we were also interested in balancing expenditure 
per pupil with achievement-based equity, the best configuration seems 
to small single-unit school. While decreased size would increase 
costs, a (logged) value of 1 on HIGHSKLS (equaivalent to 
approximately 3 high schools in a district) and a value of 13 to 15 on 
LEVELS would substantially diminish costs (Note 15). This makes 
the achievement advantage of small schools (where they are most 
needed, that is, in impoverished communities) more affordable than 
previously expected. 

This study once again corroborates the manner in which SES 
regulates the relationship of school size to school performance. The 
findings have proven to be unusually robust, which makes them 
difficult to dismiss. This study's findings with regard to ways to reduce 
school costs without increasing size are more tentative, and our 
explanations of them are more tentative as well. Nevertheless, in the 
effort to resolve the aim of achievement equity within manifest fiscal 
constraints, it seems time to consider the issue of district organization 
and school grade span configuration. 
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Notes 

1 . Evidence from a related study conducted with Alaska data 
(Huang & Howley, 1993) , which included several blocks of 
contextual, student background, and school-level process 
variables, suggests that the interaction effect may be robust. 
Using individual-level data, the interaction term remained 
significant after entry of all blocks of relevant data. 

2. The controlled vocabulary of the ERIC database includes "grade 
sp 2 m configuration" as an "identifier," but not as a "descriptor." 
Descriptors are main indexing terms and are adopted after a 
lengthy and formal deliberation; identifiers may be coined by 
any EWC clearinghouse at any time and serve as proto- 
descriptors. As of this writing, "grade span configuration," 
added in the early 1990s, had been used to index just 4 items. 

3. These are sometimes referred to as "union schools" (e.g., in the 
Southeast) or "unit schools" (e.g., in the West) schools. 

4. All independent variables originate with the Texas Department 
of Education. In particular, PCTINST is computed by dividing 
the DOE'S dollar value for instruction by "total campus budget." 
Approximately 80 percent of PCTINST, which varies among 
schools, is accounted for by teacher salaries. 

5. These three programmatic terms are included for the sake of 
model specification as control variables hypothetically 
associated with increased EPP. Our analytical focus, however, 
remains organizational rather than programmatic. One 
anonymous reviewer of an earlier draft of this article observed 
that most children eligible for special education services are not 
in full-time programs (PCTSPECL). We recognize this fact, of 
course, but for our purposes, PCTSPECL is a proxy for the 
additional cost of providing special services in a school. The 
correlation of PCTSPECL and EPP is positive, as expected r = 
+.53); S/TRATIO, however, predictably co varies with 
PCTSPECL r = -.44) and the net influence of PCTSPECL (in 
the multivariate analyses) becomes statistically nonsignificant 
when both independent variables appear in our equations. 

6. A full discussion of the use of partial derivatives in this fashion 
appears in Bickel and Howley (2000) and in Howley (1995). 

7. This is not, readers unfamiliar with economic analysis should 
note, case weighting as used in in analyses of data produced by 
oversampling, but an application of a simple weighted average 
serving as a proxy for average differences in cost, statewide, by 
educational level. The variable incorporates these norms into a 
single, school-wide metric as a control variable, once again for 
the sake of model specification. 

8. The Texas case, we think, is illustrative of the larger policy issue 
of size and grade span configuration. One of the authors 
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(Howley) has consistently argued that the ratio of total school 
enrollment to grade span is the most proper metric of school 
size. That metric, however, makes it impossible to treat the 
influence of grade span configuration separately from school 
size. Separating the two issues allows for grade span 
configurations other than the dominant 9-12 arrangement (10- 
12, 7-12, 5-12, or, indeed, K-12). 

9. In a survey of all unit schools nationally, two-thirds of 

responding superintendents indicated that their school district 
operated a single school— the K-12 unit school in question. All 
responding Texas superintendents indicated their schools were 
in this category. Among the other states, most seemed to 
maintain unit schools principally on this model. States where 
unit schools were more frequently part of multi-school districts 
included Alabama, Alaska, Louisiana, and Mississippi (Howley 
& Harmon, 2000a). 

10. That is, the moderately strong correlation did not introduce 
multi-collinearity problem, which means we can affirm that as 
SIZE increases, EPP declines, and as S/TRATIO increases, EPP 
declines. 

1 1 . The possible value combinations of the independent variables in 
the partial derivative are considerable, and so are the possible 
effect sizes that are the function of such values. In Table 7, then, 
The 12 values of the relevant independent variables chosen to 
illustrate the range of effect size variation, then, are merely 
illustrative. For another application of this sort of illustration, 
see Bickel & Howley, 2000; see also note six for reference to 
the use of partial derivatives to estimate effect sizes across this 
series of studies. 

12. In Table 7, recall that HIGHSKLS is logged, so that a value of 0 
(ln=0) is equivalent to an unlogged value of 1, indicating a 
single high school, the category to which all single-unit schools 
belong. 

13. Because we were more interested in policy matters than in the 
conditions of instruction, we did not plan for a multi-level 
analysis of students within schools; individual-level information 
was not part of our data set. 

14. We provide significance levels on the assumption that "A 
population... in a given time interval includes not only the actual 
history represented by the values that were in fact observed but 
also the potential history consisting of all the values that might 
have occurred but did not. The population so defined is 
obviously an infinite one.... This view underlies virtually all 
policy-oriented research in economics and 

econometrics" (Kmenta, 1997, p. 4). The use of significance 
levels also provides one rubric, when a study population so 
closely represents the universe, forjudging practical 
significance. In this study, we dismiss as practically 
insignificant influences that do not attain levels of statistical 
significance. 

15. Recall that e' ~ 2.72; that is, the unlogged value of ln=l is e, or 
approximately 2.72. 
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Abstract 

Higher education workers in Washington State are 
challenging the use of contingent academic labor. This 
article examines data and policies relevant to the state's 
reliance upon part-time faculty in community colleges. 

Data from the State Board for Community and Technical 
Colleges is juxtaposed with results from a survey 
completed by 20% of the part-time faculty in 14 
commimity colleges to show that most do not work part- 
time by choice. The quantitative analysis underlies a 
subsequent examination of legislative and court solutions 
pursued in Washington State. Despite significant spending 
constraints, the state shows signs of being in the national 
vanguard as it addresses contingent academic labor issues. 
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College and university educators in Washington State are 
stepping up their resistance to belt-tightening measures that increase 
reliance upon contingent academic labor. While results thus far are 
mixed, several signs suggest that the state is on the cutting edge the 
issue. Positive omens include the passage of a state initiative that 
provides cost of living adjustments for kindergarten through 
community college teachers, a 20 million dollar state down-payment 
that reduces the part-time community college teacher wage gap, a 
court challenge against the exclusion of part-time instructors from 
benefits, and, a state plan to increase the number of full-time faculty 
lines at community colleges. These initiatives constitute a ray of light 
on the otherwise darkened landscape of higher education. 

This article will first highlight the state context in which faculty 
concerns have risen. Having laid that groundwork, I report on the 
parameters within which contingent academic work now occurs within 
the state's community colleges. Next the discussion turns to how part- 
timers have mobilized and the effect they have had on policy. The 
article concludes by examining some of the challenges in the 
immediate future. 

A Financial Outlook on Higher Education in Washington 
State 



Washington's higher education faculty has become more 
assertive as the severity of restraints on state funding for education has 
increased. Washington's perennial fiscal crises have been compounded 
by unorthodox fiscal constraints. The state is one of eight that have no 
income tax. Additionally, under Initiative 601, passed in 1994, 
expenditures were capped to grow no faster than population and 
inflation combined. Because the educational constituency has grown 
faster than these limits, officials find themselves trying to fund higher 
education on the cheap. 

That the community college system forms the bedrock of the 
state's higher education infrastructure is symptomatic of these financial 
difficulties. During 1999-2000 there were approximately 125,000 
state-funded full-time equivalent community college students [FTE]. 
By contrast, the four-year colleges enrolled only 28,000 state-fonded 
freshmen and sophomores, along with 41,000 additional 
undergraduates at the upper division level. The difference in lower and 
upper classman at the four-year schools is partly made up by the 
annual inflow of approximately 1 1 ,000 new transfer students from the 
community colleges each year. Given state data indicating 27% of 
community college entrants (87, 500 students by headcount as 
opposed to FTE) intend to transfer to a four year institution, it is 
evident that the vast majority of students intending to complete a 
bachelors degree begin their higher education in Washington State via 
the two-year college system (SBCTC, 2000A). 

The funding formulas for higher education are a likely factor 
accounting for this pattern. The State Board for Community and 
Technical College's [SBCTC] 2001-03 biennium budget request 
makes the case that community colleges receive less than foiu 
thousand dollars in state funding per student compared to regional 
institutions, where per student fonding is approximately five and one 
half thousand. Funding differentials are exacerbated by the state's 
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recent policy shift enabling colleges to retain their own tuition dollars. 
Thus, in addition to the $1,500 difference in general fund revenue, 
tuition disparities increase the shortfall in per student spending at the 
community colleges to a figure between $2,500 and $3,500 (SBCTC 
2000B). 

Washington State community colleges, like those almost 
everywhere else, have consistently been under-funded relative to their 
four-year peers. The lower funding formula was one of the attractions 
of building out the community college system in the early 1970s. 
Although five new upper division campuses were inaugurated in 1 990 
in order to encourage students to complete their bachelor degrees, 
coordination has been difficult and upper level enrollment growth 
much slower than anticipated. The "seamless education" that was the 
talk of the 90s has clearly not patched the system together. One 

consequence is that the state is ranked 46 in the nation in the 
production of four-year degree holders. 

As if panaceas like "seamless education" were not bankrupt 
enough, in 1997 Governor Locke advanced the idea of the "virtual 
university." Expansion of brick and mortar education was declared 
financially infeasible and in its place he proposed on-line distance 
education as a substitute to accommodate increased enrollment. 
Patterned after the now financially plagued Western Governors 
University, Locke's proposal quickly generated opposition at the 
University of Washington as 600 faculty signed petitions rejecting the 
idea. But financial pressures continue. Within the University of 
Washington, the cause of contingent academic workers caught hold 
and teaching assistants organized to demand their own union in the 
Spring of 2000. In the fall, the Faculty Senate closed ranks behind the 
TA's and asked the administration to recognize their union. UW 
President McCormick took the bold step of reversing a long-held 
administration policy and announced an agreement in which the TA's 
and the University would jointly approach the legislature to request 
enabling legislation establishing a framework under which TA 
bargaining rights will be established and negotiations be conducted. 
However, when the state legislature failed to pass the legislation the 
University refused to grant exclusive bargaining rights to the TA s. 

The teaching assistants went out on strike during the June final exams 
and vow to jumpstart their campaign again next year. 

Statewide faculty demands for higher wages continue to heat up 
the issue of union bargaining rights at four-year colleges and 
universities. Despite the absence of state enabling legislation, a faculty 
union at Eastern Washington has now operated successfully for several 
years. However, at Central Washington University the issue has also 
been raised but amiable relations appear distant. Some University of 
Washington professors continue to press for enabling legislation, but 
the future of such legislation is now linked to the standoff with the 
teaching assistants at the institution. Even with all these issues 
percolating, it is among the part-time faculty at the state's community 
colleges that the most inequitable situations exist. 

Failing to rally forces around an anti-601 initiative, the 
Washington Educational Association sponsored, and the Washington 
Federation of Teachers ultimately endorsed, a citizen's referendum 
which guaranteed teachers from Kindergarten through Community 
College raises in line with the cost of living. In a statewide election the 
referendum passed overwhelmingly. Unfortunately, that stopgap 
measure complicated the task of closing the pay differential between 
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part-timers and full-timers because it reduced the pot of money to be 
spent raising part-time salaries. In June and July of 2001 the governor 
extended the legislative season with three special sessions to break the 
deadlocks surrounding budget issues. Despite the budget wrangling, 
part-timers have corralled another 7.5 dollars from the legislature to 
help close their pay gap. 

The Part-time Issue in Washington's Community Colleges 

Washington State is a leader in community college education. 
Whether that is something to brag about depends upon what you look 
at. The state's faculty is among the most creative in developing new 
models of teaching. With help from the Washington Center for the 
Improvement of Undergraduate Education, the state's community 
college system has successfully championed learning communities in 
which questions are investigated using teams of faculty from different 
disciplines. In another show of quality. President Clinton touted one of 
Shoreline Community College's job training programs as a national 
model. In these and other areas the state's community colleges 
demonstrate drive and originality. Lurking beyond these positive 
images are the problems created by inequities in faculty employment. 

A May 2000 report from the National Center for Educational 
Statistics makes clear that the pattern of part-time employment in 
Washington is not unique. According to their national survey, sixty 
two percent of the 255,000 instructional faculty and staff working in 
the nation's two-year schools were employed part-time. Because part- 
timers averaged less than half the course load taught by full time 
teachers (2.1 courses per semester vs. 4.5), part-time faculty instructed 
roughly 43 per cent of the systems students (NCES, May 2000, pp. 39 
and 78). Washington SBCTC data places the state almost dead even 
with this national average. While the use of part-timers in Washington 
accelerated in the early 1990s, that increase has practically halted (Best 
Practices Task Force, 1996). (Note 1) Since 1995 the percentage of 
part-timers has risen less than 1%. It appears likely that lobbying by 
part-timers was a factor in changing the trajectory of part-time 
employment. 

The magnitude of part-time faculty participation in the 
instruction of community college undergraduates forced the 1 996 Best 
Practices Task Force to admit that the adjunct system had been abused. 
Hiring exceeded the level which could be justified educationally: "[B] 
udget reductions, increased enrollment that is not fully funded, and 
similar requirements to 'do more with less' all create a powerful 
incentive for colleges to employ adjunct faculty for purely economic 
reasons— to deliver needed services within available budgets" (Best 
Practices, 1996, p 4). 

The incidence of part-time faculty is imeven across the 
community college system. Some colleges find ways to hire more full 
time faculty, just as some programs within colleges are less deeply 
affected. Overall, rural colleges are less dependent on adjuncts, largely 
because they find it difficult to recruit them. Likewise, technical 
colleges, where job training predominates, are staffed almost entirely 
by full-time instructors. 

The use of part-timers is most disproportionate within the Basic 
Skills area, particularly in English as a Second Language [ESL] 
courses. Part-time instructors taught slightly over 69% of the FTE 
course-load in Basic Skills courses throughout the state system. 
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Humanities, where part-time instructors taught 48% of the courses, 
occupied second place. In only three of eight broad classifications 
were fewer than 40% of courses taught by part-timers: These divisions 
include Mechanics and Engineering 25%; Social Sciences 36%; and 
Science 37%. 

Table 1 demonstrates that the proportion of classes taught by 
full-timers rises substantially when we remove the roughly 15% of 
classes taught during the evening, off the main campus, and those 
relying on non-state funds. One may justify this exclusion under the 
assumption that these are the arenas in which the "flexibility" of a part- 
time faculty is necessary. This exercise reduces the incidence of part- 
time instruction from 43% to 30% of FTE class instruction. However, 
breaking down totals by division continues to reveal the same patterns 
of part-time employment: The three divisions in most dependent upon 
part-time employment are, in descending order: Basic Skills (50%), 
Humanities (39%), and Math (33%). 

Table 1 

Use of Part-timers in Washington State, Fall 1999 

(For All Courses On-campus, Day, State Funds) 



Academic 

Area 


PT FTE 
Courses 
Taught 


Total FTE 

Courses 

Taught 


PT FTE 
Courses 
Taught 


Total FTE 

Courses 

Taught 


Basic Skills 


391.03 


563.51 


81.78 


161.95 




69.39% 




50.50% 




Business, 

Data 


357.88 


871.51 


136.48 


535.04 


Processing 


41.06% 




25.51% 




Humanities 


596.02 


1232.88 


351.41 


898.97 




48.34% 




39.09% 




Math 


217.22 


493.76 


120.99 


360.39 




43.99% 




33.57% 




Mechanical 


152.94 


603.3 


58.83 


424.78 


Engineering 


25.35% 




13.85% 




Public 

Service 


477.25 


1110.22 


207.4 


689.82 




42.99% 




30.07% 




Science 


145.9 


394.04 


66.23 


279,12 




37.03% 




23.73% 




Social 

Science 


157.13 


434.13 


75.96 


300.27 




36.19% 




25.30% 




Totals 


2495.37 


5703.35 


1099.08 


3650.34 




43.75% 




30.11% 





Source: Compiled from Washington State Board for Community and Technical 
Colleges Data. 
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To learn more about part-timers preferences and work history, 
the Washington Federation of Teachers surveyed faculty at 14 of the 
institutions at which it is the bargaining representative. Surveys were 
given to union representatives to distribute to all part-timers on their 
campuses. Five hundred fifty five separate surveys were returned. 
While the method of distribution and collection leaves open the 
probability of sample bias, these surveys provide a legitimate basis to 
draw conclusions when appropriate qualifications are noted. Statistical 
results must be regarded as suggestive, not as precise population 
estimates. 

In anticipation of the survey results, it is helpful to examine 
potential sources of bias. Surveys were distributed through campus 
mailboxes. However, some part-timers do not have mailboxes, while 
others do not teach at the central campus of their institution and may 
not have been reached. Although campus leaders at some colleges 
made a concerted effort to exhort their part-timers to return the 
surveys, at other campuses surveys were returned on a more casual 
basis. 

It is important to determine whether the returned surveys 
constitute a representative cross section of faculty at the colleges. In 
Table 3, we can see that some of the 8 major disciplinary categories 
used by the State Board to define subject area, such as Basic Skills and 
Humanities, are significantly over-represented. Responses in the 
Sciences and Social Sciences, however, more closely reflect the 
distribution of faculty by those areas. Given the varied response rates, 
it is probable that the survey as a whole is biased toward faculty more 
aggrieved by part-time issues. Thus, results are best interpreted as 
indicating the direction of change of employment concerns as specific 
variables change. 



Table 2 

Number and Percentage of Returns from Washington 
State Community Colleges 



* 

College 


Prime’ 

Affiliation 


% 


Headcount 


TotaP 


% 


Centralia 


14 


11% 


128 


21 


16% 


Edmonds 


81 


28% 


294 


106 


36% 


Everett 


32 


16% 


196 


39 


20% 


Peninsula 


25 


17% 


149 


26 


17% 


Pierce County 


28 


8% 


343 


34 


10% 


Seattle Central 


44 


13% 


328 


57 


17% 


Seattle North 


31 


10% 


303 


48 


16% 


Seattle South 


23 


9% 


267 


34 


13% 


Shoreline 


65 


22% 


294 


93 


32% 


Skagit Valley 


37 


19% 


195 


48 


25% 


South Puget 


38 


24% 


160 


47 


29% 


Tacoma College 


35 


13% 


269 


49 


18% 


Whatcom 


45 


29% 


157 


47 


30% 



httn://enaa.a.sii edii/enaa/v9n41 / 



3G2 



10/30/01 





EPAA Vol. 9 No. 41 Jacoby: Progress on Addressing Contingent Work Issues in Academia Page 7 of 15 



Yakima Valley 


25 


14% 


182 


26 


14% 


Overall 


523 


16% 


3265 


675 


21% 



'Faculty assigned to schools according to their stated primary affiliation 
^Percentages are calculated using state data for Fall 1999 as shown. These data are 
for all Part-time faculty, including those on contract funding, teaching at night or on 
other campuses. 

^Calculated using all data from faculty who taught at school, regardless whether they 
identified this is primary affiliation. It is appropriate to group responses by college 
when looking for college wide information but because some faculty taught at more 
than one campus, both the state's total of 3265 faculty and the survey total of 675 
involve double counts. 

*Only colleges with total response rate in excess of 10% included. 

The survey was designed to provide information on two primary 
concerns. First the survey intended to gage whether part-time faculty 
prefer greater levels of employment. Second, the survey was designed 
to permit an investigation into categories that might help us understand 
those preferences. 



Table 3 

Distribution of Faculty by Academic Field of Employment 



Employment Field 


Number Faculty 


y. of Survey 


%of State FTE 


Difference 


Basic Skills 


115 


ZUy. 


9.9% 


11.9% 


Business, Data etc 


52 


%»y. 


15.3% 


-5.5% 


Humanities 


159 


30.15« 


216% 


8.4% 


Mathematics 


52 


9.8*/< 


8.7% 


1.2% 


Mechanics/Eng'rg 


8 


1.5% 


10.6% 


-9.1% 


Pubiic Service 


78 


14.4% 


19.5% 


-5.1% 


Science 


35 


6.6% 


6.9% 


r -0.3% 


Social Science 


27 


5 1% 


7 6% 


-2.5% 



Clearly, not all part-timers desire full-time employment. 
However, the WFT survey indicates that 50% did, while an additional 
1 8% said they wanted more work than they have presently secured 
through their community college jobs. The percentage of those 
reporting dissatisfaction is thus very large. 

Table 4 indicates that the majority of faculty reported they were 
either the only wage earner in their family, or that that teaching was 
the primary source of their income. Fully 59% (n=505) of the 
individuals surveyed reported that part-time teaching was the primary 
source of their personal income. Additionally, 34% (n=174) reported 
that their earnings were the only source of income in their household. 
Within the 27% (n=136) of respondents who reported their community 
college teaching as both the only source of income in their household 
and as their primary source of income, nearly 84% said they wanted 
more work (n=21) or full-time work (n=l 14). Preferences for full time 
work were also higher when individuals were the only breadwinners in 
their household (63% compared to 50% among all survey 
respondents), and also when earnings from teaching \vere the primary 
source of individual income (also 63%). Thus a sizable group 
indicated that community college income contributed significantly to 
their livelihood and, among these, the majority indicated a desire for 
additional employment. 

Table 5 indicates that faculty prepared in traditional disciplines 
within the arts and sciences rely more heavily upon their part-time 
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teaching income, at least as indicated by their relative preference for 
full-time or increased work. Thus, survey results show that 85% of 
social science, 76% of humanities, and 74% of science faculty prefer 
more work than they presently have. By contrast, those serving in non- 
traditional academic areas, such as Public Service or Business, are 
somewhat less likely to seek greater teaching employment. 
Mathematicians, curiously, appear to fall outside the expectations for 
traditional arts and science faculty. 

Table 4 

Sources of Household Income 



I 




Not Primary 


Primary 


[wot ONLY 


331 


187 


163 


As X of Total 




33>i 


32% 


jONLY 


174 


38 


136 


i As of Total 


34X 


8X 


27% 


Total 


505 


205 


299 






41% 


59% 



Source; WFT Survey 

Responses were to the following questions; 1) Is this your primary source of 
income? 2) Is yours the only source of income in your household? 

The final point to note is that many of the faculty appear to have 
adjusted to this system as best they can. Those faculty who want to 
work full-time reported that they taught an average of 3.33 classes in 
the Fall quarter of 1999. Within this group, those who identified 
themselves as depending primarily upon their community college 
earnings averaged 3.46 classes per quarter. By contrast, those who 
indicated that they were satisfied with their teaching load reported an 
average of 2.17 class per quarter. The SBCTC, on the other hand, 
reports that average workloads are lower, and that only 45% of part- 
time faculty taught more than one course in fall 1997. While sample 
bias may account for some of this difference, the SBCTC figures, too, 
are biased reflections of overall teaching duties because they omit 
courses that were not state funded or that were outside the community 
college system altogether (SBCTC, Research Report 98-4). 

There is interest in the phenomenon known as the "freeway 
flyer," in which part-time teachers work at more than one campus to 
make ends meet. Some 248 of survey respondents reported teaching at 
two or more institutions. Among these, 90 said they taught at three or 
more colleges. This finding is at odds with SBCTC data indicating 
only 27 persons statewide taught at three or more colleges. It also casts 
doubt on the state's conclusion that only 291 faculty systemwide 
taught at two campuses. The discrepancy may be explained in two 
ways. First, the state's analysis was not designed to verify employment 
at private institutions, nor at four-year schools. Second, in addition to 
listing schools at which they were currently teaching, individuals in 
the WFT survey may have responded to the question by citing 
institutions at which they had recently taught. The state board, by 
contrast, using in-house date data could restrict its analysis to a single 
quarter. Thus the State Board concludes that freeway flyers constitute 
13% of the part-time faculty, whereas the WFT survey suggests that 
employment at multiple campuses is more common, especially when 
considered over longer employment periods. The WFT survey means 
the part-time faculty travels more, teaches more, and spends more time 
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job searching then is generally appreciated. The educational 
consequences of these patterns have not been adequately studied. 

Table 5 

Preference for More Employment Teaching by Field 



Employment 

Field 


Want Full 
Time 


Want 

More 

Work 


Content 


% Want 
More 


Basic Skills 


54 (46%) 


23 (20%) 


37 

(32%) 


67% 


Business, Data 
etc 


19(36%) 


14 (27%) 


18 

(35%) 


63% 


Humanities 


97 (61%) 


25 (16%) 


33 

(21%) 


76% 


Mathematics 


21 (40%) 


4 (8%) 


26 

(50%) 


48% 


Mechanics/Eng'rg 


4 (50%) 


1 (13%) 


3 

(37.5%) 


63% 


Public Service 


25 (33%) 


19 (25%) 


28 

(37%) 


58% 


Science 


20 (57%) 


6(17%) 


8(23%) 


74% 


Social Science 


20 (74%) 


3 (11%) 


4(15%) 


85% 



Source: WFT Survey 



The main findings derived from the WFT survey are not 
controversial. Clearly, Washington State relies very heavily upon part- 
time faculty, and officials themselves believe that this reliance is 
greater than is educationally justifiable. In investigating the problem, 
the State has reached the conclusion that it is important to reduce this 
reliance. The WFT's data suggests that, if anything, the State still 
underestimates the extent of the problem. From the vantagepoint of the 
part-time faculty member there is much to be gained by improving 
employment security. To the extent that adverse employment and 
working conditions affect the community colleges, a point which the 
State has conceded, the education students receive at community 
colleges will be advanced by converting some part-time faculty 
positions into full-time position and by improving the compensation 
package for part-timers. 

Organizing Part-timers in Washington 

Policy in Washington State has clearly been influenced by a 
number of campaigns on behalf of part-time community college 
faculty. One result, noted earlier, is that in opposition to the national 
trend involving an increased reliance upon part-time faculty, in 
Washington that trend has been ended. In addition, pay and benefit 
conditions are being raised, albeit at an inadequate pace. Much of the 
state's progress traces directly back to two legislative decisions begun 
in 1995 and 1996. In 1995 the state redefined its unemployment laws 
to establish the eligibility of part-time faculty for unemployment 
compensation. Second, and perhaps more important, the legislature 

385 



httn7/e.nan n<5ii pHii/f»nan/vQn41 / 



10/30/01 





EPAA Vol. 9 No. 41 Jac...: Progress on Addressing Contingent Work Issues in Academi Page 10 of 15 



inaugurated a Best Practices Task Force regarding part-time 
instruction. 

This task force was the legislature's response to agitation by 
part-timers that dates back, at least, to the early eighties. It wasn't until 
1990s, under Susan Levy's leadership, that the Washington Federation 
of Teachers, seriously began to champion the part-time cause. This 
transition became even more pronounced when the WFT employed 
Wendy Rader-Konofalski, a former part-timer, as the WFT legislative 
representative in Olympia. Working through the union, Rader- 
Konofalski succeeded in getting legislative priority for the issue. In 
significant measure the \^T was spurred on by Keith Hoeller and the 
Washington Association of Part-Time Faculty [WAPFAC]. This 
advocacy group worked independently, creating a second fulcrum 
upon which to pry open state policy. Through direct lobbying and 
publicity WAPFAC maintained pressure on both the legislature and 
the WFT, ensuring that the part-time issue did not die in intramural 
union politics. Together Rader-Konofalski and Hoeller— perhaps 
unwittingly— created an inside/outside strategy that kept everyone on 
their toes. Although disagreements have at times surfaced, WFT and 
WAPFAC's successor, the Washington Part-Time Faculty Association 
have worked more closely in recent years to good effect. 

The two organizations have succeeded in forging alliances with 
the Worker Center, King County's Labor Council, Seattle Union Now, 
the University of Washington's Labor Center, and the Center for a 
Changing Workplace. Together, these groups create visibility for the 
permatemp and contingent labor force issue. Over the long haul, it has 
been the efforts of rank and file part-timers that successfully muscled 
the state into appointing its Best Practices Task Force. The Task Force 
established a foundation for continued legislative action by officially 
recognizing the abuses inherent in the part-time system and 
acknowledging that these abuses arose as the consequence of financial 
pressures. While the limited use of part-timers could be justified in 
low demand disciplines, in fields were scarce expertise is needed, or 
even when colleges can not flexibly respond to scheduling needs with 
their existing full-time faculty, the Task Force acknowledged that part- 
time staffing had gone beyond these rationales. 

The Task Force found fault with a the part-time employment 
system because it provided virtually no incentive for faculty to commit 
themselves to the classroom, to provide needed service to the campus, 
department or community, and because the system utilized poor 
selection, recruitment and development tools. To remedy these 
problems the Task Force made several recommendations. First, 
academic departments should develop a written policy on the 
appropriate use of part-timers to guide their actions. Second they 
should improve the recruitment process to ensure quality part-time 
hires while improving and smoothing opportunities for transfer from 
part to full time positions. Third, the Task Force recommended that 
administrators should provide written and early employment 
commitments for part-time faculty. It also encouraged multiple quarter 
contracts, rather than quarter by quarter renewals. Other best practices 
involving evaluation, development, communication, support and 
recognition were also put on the table. 

To make earnest its support for the task force recommendations 
Earl Hale, Executive Director of the SBCTC, announced that the State 
Board would seek twenty million dollars over the 1 997 to 1 999 
biennium to address faculty issues, including part-time salary and 



httn://enaa.asii.eduyenaa/v9n4 1 / 



366 



10/30/01 




EPAA Vol. 9 No. 41 Jac...: Progress on Addressing Contingent Work Issues in Academi Page 1 1 of 15 



benefit inequities. Ultimately the state authorized a maximum of 7.7 
million dollars to address part-time issues. Following this, a number of 
specific initiatives were taken that, cumulatively, have begun to make 
a difference for part-timers. Most significantly, in 1996 the WFT 
drafted and secured legislation to ensure that part-timers that work at 
least 50% receive the medical benefits to which they were entitled. A 
clear method of calculating percentages of employment time was 
established to prevent the state from denying those claims. Summer 
benefits have remained a point of contention and are one of the 
subjects in a major court challenge now underway. On a more positive 
note, the most direct indication that the state takes the problem 
seriously was the legislature's decision, in 1999, to dedicate twenty 
million dollars to adjust part-time pay upwards. In doing so, the 
legislature abandoned language that would have settled for the 
SBCTC's goal for part-timers— 76% of full time pay— and appears to 
have adopted the \^T's goal of 100% parity. The pay adjustments 
achieved to this date still leave part-timers far from either goal, but 
state actions stand in stark contrast to years of previous neglect. In 
June 200 1 , despite a very difficult session the legislature voted another 
7.5 million dollars for pay equity. 

As a percentage FTE instruction, the use of part-timers has not 
expanded in any appreciable degree since 1995, but neither has it been 
reduced. After discussions with the union, the SBCTC created plans to 
change the part-time/full-time faculty mix by adding some 360 full 
positions statewide in the current biennium. However, that plan 
appears to have been abandoned in the light of current budget 
difficulties. Hope for conversions must now rely upon success in 
achieving pay equity, which will act to minimize the demand for part- 
timers for purely economic reasons. The cost of providing benefits 
may begin to tip incentives away from part-time hires even without 
100% pay equity. 

Conclusions 

Despite real accomplishments, ominous clouds continue to mark 
the sky. As always, money is extremely tight in the state capitol, 
Olympia, and the part-time situation has been complicated by new 
state initiatives, one limiting taxes and another increasing pay for 
teachers from kindergarten through community college. In this fiscal 
environment nothing is certain. 

On the other hand, pressured by law suits, lobbying, and public 
relations campaigns, Washington's SBCTC appears poised to resolve 
the situation, if for no other reason than to avoid costly liability. The 
prospect of an expensive court suit related to contingent work practices 
has grown since December 12, 2000, when the Vincainzo Case against 
Microsoft was settled. To resolve that suit, Microsoft consented to a 97 
million-dollar payment to permatemp workers who claimed they were 
wrongfully denied benefits the company provided to its other 
employees. At the behest of Keith Hoeller's WPTFA the law firm that 
represented those plaintiffs, Bendich, Staughbaugh and Strong, is now 
arguing in a separate case that part-time community college faculty are 
being denied benefits they rightfully deserve. One irony is that this suit 
would have no little basis in law if the state had not acquiesced when 
the WFPTA and the WFT pressed for, and secured, best-practice 
legislation in the mid-nineties. The subsequent 1996 WFT bill spelled 
out the method by which part-timer's eligibility to participate in benefit 
plans was to be determined. The new lawsuit seeks retroactive faculty 
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benefits for up to twenty years, during which time the state allegedly 
calculated hours erroneously so as to deprive part-timers of their 
pension and health benefits. 

In an interim decision, Judge Steven Scott has determined last 
year that faculty teaching 50% or more are entitled to summer health 
benefits if they work at all at during that period. If complied with this 
interim decision may conflict with another high priority part-time 
concern: the ability to collect unemployment benefits. In particular, 
many part-timers desire unemployment compensation during summer 
and other times when colleges fail to provide them with classes to 
teach. By securing summer benefits, the claim of temporary 
employment may be weakened as part-timers begin to look more like 
full time faculty, for whom a nine-month contract is presumed to be 
full time yearly employment. Perhaps the ultimate test of the success 
of the part-time movement in Washington State will come when part- 
timers are treated well enough that they will be able to choose between 
the reasonable assurance of multi-quarter contracts with benefits and 
unemployment compensation during quarters when they don't teach. In 
April of 2001 the WFT secured a victory that should ease 
unemployment claims. The bill declares that part-time employment 
offers contingent upon enrollment, funding, or scheduling does not 
constitute reasonable assurance of employment. 

In the meantime the law firm of Frank and Rosen is pressing yet 
another case arguing that the state's method of paying part-timers is 
seriously flawed. Presently, not only does the state not provide 
reasonable assurance of continued employment, the plaintiffs in this 
case claim, instead, the state misstates the employment relationship 
altogether. The plaintiffs argue that community because colleges pay 
part-timers only for each class-contact hour, the state violates its own 
minimum wage and overtime laws. Although the case faces a variety 
of obstacles, it constitutes one more pressure point toward the 
implementation of the best practices that enumerated in 1996. 

The state continues to show modest incremental leadership in 
slowly tackling the worst of the contingent labor practices in 
academia. Perhaps the greatest danger on the horizon is degree to 
which different elements of the education community are increasingly 
being pitted against one another for sparse funds. The fact that the 
legislature provided financial relief for part-time faculty, but refused to 
pass enabling legislation for the teaching assistants at the University, 
suggests something of the constrained choices facing the higher 
education community. 
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1 . A legislatively appointed Task Force reported that the use of 
part-timers had increased 6 percentage points, from 42 to 48% 
of FTE between 1990 and 1995. It should be noted that the Task 
Force Report apparently included full time faculty who 
moonlight additional courses for extra income. Thus, 5% of 
these 48% are not part-timers. 
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Abstract 

In 1998 Massachusetts began state-sponsored, annual 
achievement testing of all students in three public school 
grades. It has created a school and district rating system for 
which scores on these tests are the sole factor. It proposes 
to use tenth-grade test scores as a sole criterion for high 
school graduation, begiiming with the class of 2003. The 
state is treating scores and ratings as though they were 
precise educational measures of high significance. A review 
of tenth-grade mathematics test scores from academic high 
schools in metropolitan Boston showed that statistically 
they are not. Community income is strongly correlated with 
test scores and accounted for more than 80 percent of the 
variance in average scores for a sample of Boston-area 
communities: 
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Schools Inside Route 128, Except Boston 
Pfedicted Score = 197.4+ 1.46 Income (1989, SIOOOs) 




Once community income was included in models, other 
factors-including percentages of students in disadvantaged 
populations, (Note 1) percentages receiving special 
education, percentages eligible for free or reduced price 
lunch, percentages with limited English proficiency, school 
sizes, school spending levels, and property values— all 
failed to associate substantial additional variance. Large 
uncertainties in residuals of school-averaged scores, after 
subtracting predictions based on community income, tend 
to make the scores ineffective for rating performance of 
schools. Large uncertainties in year-to-year score changes 
tend to make the score changes ineffective for measuring 
performance trends. 
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Section 1: Background 

A. State Testing in Massachusetts Public Schools 

The most recent form of state testing in Massachusetts public schools 
is the Massachusetts Comprehensive Assessment System (MCAS), a 
set of achievement tests sponsored and produced by the Massachusetts 
Department of Education and administered in the spring of years 
beginning in 1 998 (see Appendix 1 and Bolon, 2000). These tests have 
four sections: English language arts, mathematics, science and 
technology, and history and. social science. For the years 1998-2000, 
tests were administered in grades 4, 8 and 10. Begiiming in April, 

2001, the former grade 4 test sections were divided between grades 4 
and 5; new tests have been added in grades 3, 6 and 7. 

MCAS tests are loosely timed and include questions in multiple choice, 
short answer and extended answer formats; they are provided in 
English and Spanish. All public school students are required to take 
MCAS tests; there are no "opt-out" provisions. Students taught in 
parochial schools, other private schools, home schooling and out-of- 
state schools are not required to take or pass MCAS. For 1998 through 
2000, the Department of Education has published test questions used 
for scoring approximately six months after test administration (see 
Mass. DoE, 2000g, for example). It has produced new test forms each 
year. According to current plans, starting with the class of 2003 
minimum scores on the English language arts and mathematics 
sections will be required to graduate from high school (Mass. DoE, 
1999f) and to enroll at state colleges, except for MCAS test preparation 
courses at two-year colleges. 

The Massachusetts Department of Education publishes MCAS results 
as scaled scores in a range of 81 scale points, using the integers 200 
through 280 (Mass. DoE, 2000h). The Department assigns labels it 
calls "performance levels" to four scaled score intervals (Mass. DoE, 
1998b) and currently considers 220 the minimum passing scaled score 
on all test sections (Mass. DoE, 1999f). The Department has not fully 
disclosed details of assigning scale factors, assuring consistent scores 
across test forms or assuring that scores quantitatively reflect published 
academic standards. It has not published distributions of either raw 
scores or scaled scores. It has released limited information about test 
design and properties in "technical reports" for the years 1998 (Mass. 
DoE, 1999c) and 1999 (Mass. DoE, 2000i). After an independent 
analysis of score averages by "racial" and "ethnic" categories for 1998 
(Uriarte and Chavez, 1999), the Department published its own analysis 
of this type for 1 999 (Mass. DoE, 2000c). 

Massachusetts tests appear to rank near the high end of state 
achievement tests in difficulty, although failure rates are lower than 
those for some tests used in Arizona and Virginia. As in several other 
states, substantially higher failure rates are found on mathematics than 
on language tests in high schools. Since the tenth-grade version of the 
mathematics test section sets the graduation threshold for most 
students, its scores have been used as subjects for these studies. 
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B. Schools in the Boston Metropolitan Area 

Metropolitan Boston is diverse. Besides the City of Boston it includes 
many smaller municipalities, all operating their own school systems. 
These studies consider communities inside Route 128, a highway 
designed in the late 1 940s (now an Interstate), enclosing areas within 
about 9-12 miles from Boston's government center—that is, Boston and 
its inner and middle suburbs. They share a public transit system, 
several public and private utilities, and an economy dominated by 
service industries. They include poverty areas, concentrations of 
wealth, middle-income communities, prosperous suburban towns, a 
few medium-sized cities and one large city. The areas are bounded by 
the Massachusetts Bay and Atlantic Ocean to the east, Salem and 
Peabody to the north, Waltham and Newton to the west, and Braintree 
and Quincy to the south (see Metropolitan Area, 1997). 

Schools in the Boston metropolitan area are also diverse. These studies, 
focusing on testing for graduation, consider only high schools. While a 
majority of the area's population of high-school age attends public 
schools, (Note 2) a substantial proportion attends parochial schools that 
began to be established by the Roman Catholic cWch more than 1 50 
years ago. A smaller fraction is taught in other private schools or 
though home schooling. The Metropolitan Council for Educational 
Opportunities (METCO), founded in 1963, uses state funding to help 
send over 3,000 Boston minority students to suburban schools (Orfield, 
et al., 1997). 

Within public school systems there is also substantial diversity. All 
communities must support regional "vocational," "technical" and 
"agricultural" high schools. Some such schools began as "manual 
training" schools in the 1800s. Some communities have closed their 
local vocational schools; some have merged them with their academic 
schools. These studies look in detail only at academic schools, because 
the curriculum of vocational schools is substantially different and is not 
designed to prepare students for MCAS tests, an issue of controversy 
(Nicodemus, 2000). For purposes of these studies there are difficulties 
with a few communities, including Cambridge, Quincy, Revere and 
Waltham, which provide vocational education in the same facilities as 
academic programs (Mass. DoE, 2000f). I chose to include such 
schools in these studies while noting their special characteristics. 

Several communities also operate experimental schools, including 
"pilot schools" in Boston and "charter schools" in several communities 
(Partee, 1997, and Wood, 1999), as regulated under the Massachusetts 
Education Reform Act of 1993. All that offer ninth-grade curriculum 
and above are smaller than the regular academic schools. These schools 
provide motivational environments and may exercise indirect forms of 
student selection that differentiate them from other public schools. 
Primarily because of concerns about small sample sizes, schools with 
fewer than 100 students per grade are excluded from these studies. So 
far no experimental school is that large. 

The City of Boston presents a unique situation. Of its large academic 
high schools, three are exam schools: the Boston Latin School 
(founded in 1635) and the more recent Latin Academy (formerly Girls 
Latin) and O'Bryant School of Mathematics and Science (formerly 
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Boston Technical). These draw away many Boston students who tend 
to score well on achievement tests, promoting a longstanding social 
stratification in Boston schools. Over half the students at Boston Latin 
come to it from parochial and other private schools (Daley, 1997); 
some say those students would not otherwise attend Boston schools. 
However, other public school students who are not admitted leave the 
district for high school. Starting in 1975, because of federal court 
orders to desegregate, exam school admission policies included a 35 
percent set-aside for African American and Latino students, maintained 
voluntarily after 1987. As a result of another federal court decision 
(McLaughlin, 1996), this approach was weakened in 1997. As with 
academic schools that provide vocational education, the Boston exam 
schools are included in these studies, but their special characteristics 
are noted. 

C. Statewide MCAS Test Results 



Table 1-1 shows that statewide, tenth-grade MCAS test scores have 
remained nearly constant in English language arts and in science and 
technology for the years 1998-2000, while scores in mathematics have 
risen substantially (Mass. DoE, 2000h). (Tenth-grade tests were not 
given in history and social science.) 



Table 1-1 

MCAS Statewide Results, 1998-2000 



Section 


Year 


Average 


% Level 


% Level 


% Level 


% Level 






4 


3 


2 


1 


English 


2000 


229 


7 


29 


30 


34 


English 


1999 


229 


4 


30 


34 


32 


English 


1998 


230 


5 


33 


34 


28 


Math. 


2000 


228 


15 


18 


22 


45 


Math. 


1999 


222 


9 


15 


23 


53 


Math. 


1998 


222 


7 


17 


24 


52 


Science 


2000 


226 


3 


23 


37 


37 


Science 


1999 


226 


3 


21 


39 


38 


Science 


1998 


225 


1 


21 


42 


36 



Table 1-1 reflects Massachusetts Department of Education practice of 
recording students absent for a test section as scoring 200 and in Level 
1 , the lowest level (Mass. DoE, 2000h, Table 1 1 footnote). An 
undisclosed fraction of students were excluded from testing because of 
special conditions and are not counted in this report; others may have 
been provided with an alternative assessment. As currently planned, 
students in Level 1 will be ineligible to graduate from high school as of 
2003. Based on this record of scores, about half of all Massachusetts 
public school students are at risk of being denied graduation. 



The labels of the four "performance levels" designated by the 1998 
Board of Education(Note 3) for reporting MCAS results are: 
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• Level 4, "Advanced" 

• Level 3, "Proficient" 

• Level 2, "Needs Improvement" 

• Level 1, "Failing" 

Although these levels have qualitative descriptions (Mass. DoE, 
1998b), there are no quantitative links to levels of achievement 
specified in academic standards; content of standards has not been 
prioritized; nor have standards been promulgated through state 
regulations, as anticipated by law. (Note 4) Although Massachusetts 
law requires "competency determination" in mathematics, science and 
technology, history and social science, foreign languages and English, 
(Note 5) Massachusetts laws and regulations continue to require only 
US history and physical education as subjects of instruction. 
Massachusetts tries to set legal standards for learning indirectly (Note 
6) through MCAS tests, procedures to set scale factors, and regulations 
for minimum scaled scores. It lacks corresponding legal commitments 
for instruction. It has made major changes to "curriculum frameworks" 
every few years (Mass. DoE, 2000k) emd has not provided reasonable 
spans of time for instruction to catch up before using revised 
"curriculum frameworks" as a basis for revised MCAS tests. Its 
teachers, parents and students cannot find out exactly what must be 
learned in order to meet minimum standards for high school 
graduation. The 1 993 Education Reform Act left several such 
problems; few have been addressed yet by the Massachusetts 
legislature or Board of Education. 

Students with disabilities (also called special education students) and 
students with limited English proficiency (LEP students) tend to 
receive drastically lower MCAS scores than other students, although 
some students with disabilities are soon to be provided alternate 
assessments (Mass. DoE, 20001), and some LEP students have been 
able to take tests in Spanish (Mass. DoE, 2000d). The Department of 
Education has not disclosed the fractions of students who are eligible 
for or have utilized its special accommodations, although it has 
published statewide summary data using these student categories 
(Mass. DoE, 2000i, Table 14.5). Most minority students also receive 
lower scores than other students. The Department of Education has 
published 1999 statewide and district summary data for students 
categorized as "African American / Black," "Asian or Pacific Islander," 
"Hispanic / Latino," "Native American," "White" and "Mixed" (Mass. 
DoE, 2000c, Tables 5-10). As previously noted, most students in 
vocational programs receive lower MCAS scores than students in 
academic programs; this can readily be shown for the state's more than 
30 vocational, technical and agricultural high schools (Appendix 2). 

Based on the sources of information cited. Table 1-2 shows statewide 
impacts of these known risk factors on average 1 999 tenth-grade 
mathematics scores and rates of failure. 

Table 1-2 

MCAS 1999 Grade 10 Math Scores by Risk Factors 
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Category 


Average Score 


Percent Failing 


All students 


222 


53 


Students with disabilities 


203 


92 


Limited English proficiency 


206 


84 


African American 


209 


80 


Hispanic / Latino 


208 


85 


Native American 


211 


77 


Vocational, technical, agricultural 


210 


78 



Source ot data: Mass. UoE, 2(JUUi 



The Department of Education has not reported scores classified by 
other potential risk factors on which it collects information. These 
include: Gender of students, Tests taken in Spanish or as alternate 
assessments, Free or reduced price lunches, as indicators of poverty, 
Schools with large class sizes, especially in early grades. Students 
retained below grade or placed below grade level. Teachers who lack 
certification in their subjects of instruction. 

There is also little published information about combinations of risk 
factors. However, since the Department of Education lists regional 
vocational, technical and agricultural schools as separate districts in its 
reports of MCAS results, it is possible to use their categories of 
minority students (Appendix 2). For those schools for which categories 
are reported, results are shown in Table 1-3. 



Table 1-3 

MCAS 1999 Grade 10 Math Scores by Combined Risk Factors 



Combined Category 


Average Score 


Percent Failing 


Vocational + African American 


203 


97 


Vocational + Hispanic / Latino 


205 


95 



While the results in Table 1-3 are not strictly comparable with Table 1- 
2, because not all the schools and categories can be found in published 
data, they indicate that factors can combine to worsen the scores of 
students with more than one risk factor. 



D. Test Score Studies 

Recent studies question assumptions that "high-stakes" tests like 
MCAS can provide valid measures of either student achievement or 
school performance, showing gains on them that are not matched by 
gains on other tests for closely related educational content (Haney, 
2000, and Klein, et al., 2000). Political environments of "high-stakes" 
tests create heavy pressure to improve scores, regardless of underlying 
educational progress. For "low-stakes" tests aimed at measuring long- 
term trends, like those of the federal NAEP, it has been shown that 
"family variables explain most of the variance across scores in 
states" (Grissmer, et al., 2000, Chapter 9). Individual and longitudinal 



httn ;//eDaa. asu.edu/ enaa/v9n42/ 



378 



10/30/01 





EPAA Vol. 9 No. 4...: Significance of Test-based Ratings for Metropolitan Boston School Page 8 of 60 



studies demonstrate strong influences of parenting practices, family 
structure, parent education and degrees of poverty on cognitive 
development (for example. Smith, et al., 1997). Other longitudinal and 
cross-sectional studies show cumulative responses of test scores to 
educational environments (for example, Phillips, et al., 1998, and 
Ferguson, 1998). However, the data generally available for test score 
research fail to capture much of the critical information needed to 
understand development of cognitive abilities and educational 
achievement in the settings of public schools. 

MCAS test scores have already been the subject of several attempts to 
explain, predict or interpret them (Mass. DoE, 2001, Gaudet, 2001, 
Tuerck, 2001a, and Tuerck, 2001b). These prior MCAS test score 
studies fall into three main categories: 1) Trends studies of year-to-year 
and multi-year changes; 2) Effects studies involving social factors for 
the population; 3) Effects studies involving operating factors for the 
schools. 

Research on scores from school-based standard tests suggests that 
many such studies are likely to yield results of low significance. 
Grissmer, et al., 2000, among others, show that: 

• Real year-to-year changes in average student performance, as 
assessed by conventional tests, are relatively small; they can 
easily be masked by statistical uncertainties. 

• Social factors are strongly associated with test scores. 

• Self-reported social information tends to have high error and 
omission rates. 

• Census and other community-based social information often 
includes confounding factors that require adjustment to reflect 
the households for a school population. 

• Uncategorized school spending is only weakly associated with 
test scores. 

The MCAS test score studies cited use scores and statistical data to 
estimate the performance of schools or districts according to simple 
formulas, unsupported by other evidence. They frequently present 
results in a table that is ranked or can be ranked like the teams in a 
sports league. The "league table" approach to presenting such results 
begs the question of whether the ordering of schools or districts and the 
differences in performance estimates have educational significance, 
that is, whether such rankings may instead be largely matters of chance 
or be associations with factors other than school performance. This 
article presents a trends study and an effects study I conducted to 
explore the significance that can be associated with such results. 

E. Sources of Data 

The school characteristics used in these studies are taken from 
information reported by public schools to the Massachusetts 
Department of Education for 1999 and published by the Department 
(Mass. DoE, 2000f). MCAS test scores summarized by schools are 
from 1998-2000 Department reports (Mass. DoE, 2000h). Other 
information is published by the Department for school districts, 
including program budgets and percentages of special education 
students. Information for census tracts and communities is available 
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from the US Bureau of the Census and other sources. Data analysis for 
these studies focuses on information associated with individual schools 
because aggregate information for school districts or general 
populations can mask school characteristics. Data used in these studies 
are reproduced in Appendix 3 and Appendix 4; interested readers can 
confirm them at the sources and can repeat these studies or perform 
other analysis with them. 

The Department of Education and the school districts collect other 
potentially useful information that is not currently published. Of 
particular interest are data on class size and teacher preparation. Recent 
research has shown significant association of educational achievement 
as measured by "low-stakes" tests with small class size in elementary 
schools (Nye, et al., 1999, and Krueger, 1999) and with teacher 
certification and education (Darling-Hammond, 2000), after 
adjustments for student backgrounds. Studies of the development of 
cognitive abilities cast doubt on whether other information currently 
published by government sources about population and economic 
characteristics in large geographical areas would substantially improve 
the understanding of test scores. 

A. Trends Study of Variability 

This study considers 47 academic high scho^ols in 32 metropolitan 
Boston communities through the average tenth-grade MCAS 
mathematics test scores recorded for years 1998-2000. Achievement 
tests in mathematics typically require substantial skill at language 
interpretation (see, for example, Gipps and Murphy, 1994, Chapter 6, 
p. 183). Haney, 2000, in a study of another state, found stronger 
correlations of state mathematics test scores with grades in English 
than with grades in math. As previously noted, the tenth-grade 
mathematics test is used in this study of significance because it sets a 
graduation threshold for most students. 

Test boycotts have been organized by students in several schools each 
year (Steinberg, 2000), involving 10 to 31 percent of students in 19 
cases out of the 141 test samples. To be able to compare average scores 
of schools more accurately, the average scores reported by the 
Department of Education have been adjusted by removing the scores of 
200 that were assigned to students who did not take the test, averaging 
only scores of students who participated. 

Table 2-1 shows changes in schools' average scores (Appendix 3) 
between 1998 and 1999 and between 1999 and 2000, expressed in units 
of scale points and of standard deviations. 



Table 2-1 

MCAS Grade 10 Math Test Score Changes by School, 1998-2000 





Changes 1998- 
1999 


Changes 1999- 
2000 


City or 
Town 


High School 


Points 


Delta 


Points 


Delta 


Arlington 


Arlington 


4 


3 


5 


-1 


Belmont 


Belmont 


-2 


-4 


4 


-2 


Boston 


Boston High 


1 


0 


6 


0 
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Boston 


Brighton 


2 


1 


3 


-4 


Boston 


Charlestown 


-1 


-2 


4 


-2 


Boston 


Dorchester 


-2 


-3 


1 


-5 


Boston 


East Boston 


1 


0 


5 


-1 


Boston 


Hyde Park 


0 


-1 


1 


-5 


Boston 


Jeremiah Burke 


4 


3 


3 


-3 


Boston 


South Boston 


0 


-1 


5 


-1 


Boston 


The English 
High 


1 


0 


4 


-2 


Boston 


West Roxbury 


3 


2 


5 


-1 


Boston Exam 


Boston Latin 


8 


10 


8 


3 


Boston Exam 


Latin Academy 


3 


2 


19 


17 


Boston Exam 


O'Bryant 

Science 


4 


4 


8 


3 


Braintree 


Braintree 


-1 


-3 


13 


10 


Brookline 


Brookline 


2 


1 


5 


-1 


Cambridge 


Rindge & 
Latin* 


-2 


-5 


-1 


-10 


Chelsea 


Chelsea 


1 


0 


3 


-4 


Dedham 


Dedham 


1 


0 


11 


6 


Everett 


Everett* 


5 


5 


-3 


-12 


Lexington 


Lexington 


-1 


-3 


5 


-1 


Lynn 


Classical 


0 


-2 


7 


1 


Lynn 


English 


1 


0 


11 


7 


Malden 


Malden 


2 


1 


4 


-3 


Marblehead 


Marblehead 


-5 


-7 


9 


3 


Medford 


Medford* 


2 


1 


7 


1 


Melrose 


Melrose 


1 


0 


2 


-5 


Milton 


Milton 


-1 


-3 


4 


-2 


Newton 


North* 


-3 


-6 


9 


5 


Newton 


South 


2 


1 


10 


5 


Peabody 


Veterans* 


1 


0 


6 


0 


Quincy 


North Quincy 


3 


2 


7 


1 


Quincy 


Quincy* 


-2 


-4 


8 


3 


Revere 


Revere* 


2 


1 


6 


0 


Salem 


Salem* 


1 


0 


4 


-2 


Saugus 


Saugus 


4 


3 


7 


1 


Somerville 


Somerville* 


-1 


-3 


2 


-5 


Stoneham 


Stoneham 


-6 


-8 


8 


2 


Swampscott 


Swampscott 


16 


17 


-1 


-8 


Wakefield 


Memorial 


-3 


-5 


4 


-2 


Waltham 


Waltham* 


-1 


-3 


8 


3 


Watertown 


Watertown 


10 


10 


2 


-4 


Weymouth 


Weymouth* 


0 


-2 


5 


-1 
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Winchester 


Winchester 


-1 


-3 


6 


0 


Winthrop 


Winthrop* 


5 


4 


4 


-2 


Woburn 


Woburn 


0 


-2 


11 


7 



school providing vocational education 
Source of data: Appendix 3 



A "delta" is a change in scale points between two years, minus the 
average change for the year span, divided by a standard error of the 
change that is estimated from test reliability and number of 
participants. The uncertainty for one test score is based on an average 
standard error of 6.7 scale points, from reliability estimated by the 
Department of Education for the tenth-grade mathematics test of 1998, 
using randomized split-half comparisons (Mass. DoE, 1999c). The 
variance ofan average score for a school is estimated by the square of 
this quantity, plus variance contributed by roundings, divided by the 
number of test participants. 

A delta expresses the change for a particular school beyond the average 
change, in units of estimated standard errors. When standard errors are 
accurately estimated, deltas outside +/- 2 are statistically significant at 
the p< .05 level, but here about half of the schools in both year spans 
have deltas well outside this range. Especially large deltas were those 
shown in Table 2-2. 



Table 2-2 

Large Changes in MCAS Grade 10 Math Test Scores, 1998-2000 



High School 


Year Span 


Delta 


Boston Latin 


1998-1999 


+10 


Latin Academy 


1999-2000 


+17 


Braintree 


1999-2000 


+10 


Rindge & Latin* 


1999-2000 


-10 


Everett* 


1999-2000 


-12 


Swampscott 


1998-1999 


+17 


Watertown 


1998-1999 


+10 



* school providing vocational education 
Source of data; Table 2-1 



Statistically large deltas occur about ten times as frequently as they 
would by chance, were test reliability the only factor in standard errors. 
Since deltas adjust for changes in test difficulty and in average efforts 
at teaching, learning and test preparation, there appear to be other 
factors affecting test score changes that differ from school to school 
and from year to year. However, the Department of Education has not 
published any studies on other variability factors. 

Table 2-3 reflects data for all 47 schools included in this study, 
showing the average score changes in scale points for the 1998-2000 
years, weighted by numbers of test participants. 

Table 2-3 

Average Changes in MCAS Grade 10 Math Test Scores, 1998-2000 
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Year 

span 


Average point 
change 


Standard error from test 
reliability 


1998- 

1999 


+1.3 


0.1 


1999- 

2000 


+5.9 


0.1 



Source ot data: fable 2-1 



The average score change for all schools would be highly significant 
for both year spans at the p<.0001 level and better if test reliability 
were the only factor in variability. Scatterplots of score changes in 
Figure 2-1 and Figure 2-2, showing deltas for a span versus average 
scores in 1998, do not indicate strong relationships but show possible 
outliers; they are schools noted in Table 2-2. These plots of year-to- 
year changes provide a picture of the variability in school-averaged test 
scores, which is much greater than estimates based on test reliability 
and number of test participants. Evaluating year-to-year point score 
changes, excluding three cases that have deltas beyond -i-/- 10 as 
outliers, yields a practical measurement of variability for school- 
averaged MCAS test scores. 

Figure 2-1: Changes in MCAS Grade 10 Math Test Scores, 1998- 

1999 
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Figure 2-2; Changes in MCAS Grade 10 Math Test Scores, 1999- 

2000 
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Standard deviations of school-averaged score change, less average 
score change for a year span for all schools, are 2.9 scale points for 
1998-1999, 3.1 scale points for 1999-2000 and 3.0 scale points for both 
spans combined. These are about five times larger than the uncertainty 
estimated on the basis of test reliability. As one might expect, 
scatterplots of score changes in scale points versus school sizes show 
greater variability for smaller schools. An analysis found that an 
equivalent standard error in scale points for school averages of grade 
10 MCAS mathematics test scores can be estimated as the constant 33. 
divided by the square root of the number of test participants. This 
estimate of standard error combines contributions from test reliability 
with random variations in student mental alertness, student 
backgrounds and school performance. For typical metropolitan Boston 
schools from these studies, school-averaged test score gains, losses or 
differences of less than about five scale points are not statistically 
significant at the p<.05 level. 

Average mathematics score increases from 1998 to 2000 for all schools 
in these studies combined substantially exceed the rates of change that 
other studies have found to reflect genuine and sustainable 
improvements in learning. Based on test reliability as a measure of 
variance, the change is more than 20 standard deviations; based on 
calculated variance, it is more than four standard deviations. As noted, 
other tenth-grade MCAS tests showed little score change. Statistical 
magnitudes of mathematics test score changes strongly suggest causes 
other than or in addition to improvements in learning. Anecdotal 
accoimts report heavy efforts at test preparation in some schools, but 
the general upsweep in scores indicates that the 2000 mathematics test 
may also have been significantly easier than corresponding 1998 and 
1999 tests. Sparse published information from the Department of 
Education about test calibration makes this issue difficult to trace. 
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B. Effects Study Involving Social Factors 

This study, like the trends study in Section 2A, considers 47 academic 
high schools in 32 metropolitan Boston communities, through average 
tenth-grade MCAS mathematics test scores as recorded for 1999. 
School-specific social factors considered in this study (Appendix 4), as 
of 1999, are listed in Table 2-4. 



Table 2-4 

School-specific Factors for MCAS Grade 10 Math Test Scores 



Factor 


Range of values 


A. School population, average per grade 


140-495 


B. Percent African American 


0.5-86.0 


C. Percent Asian or Pacific Islander 


0.3-31.8 


D. Percent Hispanic / Latino 


0.2-63.7 


E. Percent limited English proficiency 


0.0-45.5 


F. Percent free or reduced price lunch 


1.6-64.3 


G. Percent reduction, grades 9+10 to 1 1+12 


12.9-47.0 



Source of data: Appendix 4 



The factors in Table 2-4 were used as independent variables in linear 
models for a dependent variable of 1999 school-averaged tenth-grade 
MCAS mathematics test scores. (Note 7) Residuals from the models 
are considered as estimators of school performance. Variances emd 
error estimates are calculated by conventional multivariate methods 
(Bevington and Robinson, 1992). 

For 1999, MCAS tests were, in the parlance of US public schools, a 
"medium-stakes" enterprise, associated with some indirect social 
pressures but no hazardous consequences for students. No current high 
school students stood to be denied graduation because of test scores, 
although summaries of test scores by districts and schools were being 
published. This was the second year of regular testing. It had been 
preceded by a year of trial testing, following three years of curriculum 
specification and test development. 

Note that grade size reduction, calculated as a percent decrease in 
grades 1 1 and 12 school population as compared with grades 9 and 10, 
is not identical with dropout rate. While dropout statistics are available, 
as in other states they are compromised by lack of consistent 
longitudinal data for educational outcomes. Grade size reduction 
simply indicates that, for whatever reasons, later year high school 
grades are smaller than earlier year grades. When substantial, it 
suggests that many students do not graduate in a normal time sequence. 
(Note 8) Values in about +/- 5 to +/- 8 percent ranges will be typical of 
fluctuations for schools of these sizes with stable area boundaries and 
populations and very low transiency, retention and dropout rates, 
according to Poisson statistics. 

The Department of Education publishes only district information about 
budgets and special education. Without an accurate means to apportion 
such measurements to high schools (and in four communities to 
specific high schools), they have been excluded from consideration. 
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There is considerable variation. District percentages of special 
education students ranged from 11.1 to 25.5 percent of all students and 
grades in 1999. Annual district spending reported for all regular 
education programs ranged from $3,986 to $9,251 per student in 1999. 
Even within this small group of communities, the Education Reform 
Act of 1993 failed to equalize school spending. 

Factor distributions and correlations for this model are shown in Figure 
2-3, a matrix of histograms and scatterplots with unweighted, 
unconstrained lines of best fit. 

Figure 2-3; Factor Distributions for MCAS Grade 10 Math Test 
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Although there are strong associations in Figure 2-3, such as "Percent 
Hispanic / Latino" with "Percent limited English proficiency," there are 
no factors so highly correlated as to be entirely redundant. The 
numerical correlation matrix is presented in Table 2-5, corresponding 
to the matrix of plots in Figure 2-3, with all values beyond about +/- 
0.5 significant at the p<.05 level. 

Table 2-5 



Factor Correlations for MCAS Grade 10 Math Test Scores 



Factor 


A 


B 


C 


D 


E 


F 


G 


A. School population, average per 
grade 


1.00 


-.11 


.24 


.02 


-.03 


-.01 


.16 


B. Percent African American 


-.11 


1.00 


.07 


.56 


.80 


.80 


.28 


C. Percent Asian or Pacific Islander 


.24 


.07 


1.00 


.08 


.13 


.23 


.27 


D. Percent Hispanic / Latino 


.02 


.56 


.08 


1.00 


.77 


.84 


.65 


E. Percent limited English 
proficiency 


-.03 


.80 


.13 


.77 


1.00 


.88 


.50 


F. Percent free or reduced price 


-.01 


.80 


.23 


.84 


.88 


1.00 


.60 
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lunch 
















G. Percent reduction, grades 9+10 
to 11+12 


.16 


.28 


.27 


.65 


.50 


.60 


1.00 


Sources ot data: Appendix 4 


Stal 


istica model 



Some of the correlations in Table 2-5 are strong enough that multiple 
regression coefficients are likely to be unstable. Therefore a model was 
developed in stages, examining factors for significance. 

The full model from the factors in Table 2-4 was first evaluated with 
weights proportional to numbers of test participants. It yielded two strong 
factors with low correlation (C and E): "Percent Asian or Pacific Islander" 
at p<.Q2, with a positive coefficient, and "Percent limited English 
proficiency" at /?<.002, with a negative coefficient: Factors for school 
population and percent grade reduction had particularly small coefficients 
and low significance. They were removed, and a model with the remaining 
five factors then associated 67 percent of the variance and produced the 
factor weights shown in Table 2-6. 



Table 2-6 

5-Factor Model for 1999 MCAS Grade 10 Math Test Scores 



Factor 


Coefficient 


Standard Error 


Intercept, for all factors zero 


229.4 


1.936 


B. Percent Afi’ican American 


0.047 


0.104 


C. Percent Asian or Pacific Islander 


0.347 


0.154 


D. Percent Hispanic / Latino 


-0.002 


0.183 


E. Percent limited English proficiency 


-0.637 


0.217 


F. Percent free or reduced price lunch 


-0.174 


0.157 



Sources ot data: Appendix 3, Appendix 4, Statistica model 



With model factors in Table 2-6, high factor weight and significance found 
in other studies for percentages of African American or Latino students 
disappear. Both factors have small coefficients and low significance. 
Statistical weight that might have been attached to these factors instead 
follows cultural and economic factors: "Percent limited English 
proficiency" and "Percent free or reduced price lunch." As an experiment, 
the model was rerun with the latter factors removed; only 57 percent of the 
variance was associated, and factor weights became those shovra in Table 
2-7. 



Table 2-7 

Racial and Ethnic Model for 1999 MCAS Grade 10 Math Test Scores 



Factor 


Coefficient 


Standard Error 


Intercept, for all factors zero 


230.0 


2.107 


B. Percent African American* 


-0.221 


0.068 


C. Percent Asian or Pacific Islander 


0.219 


0.156 


D. Percent Hispanic / Latino* 


-0.435 


0.114 



Sources of data: Appendix 3, Appendix 4, Stkistica model 



In Table 2-7, two "racial" or "ethnic" factors (marked *) have become 
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significant at a jo<.05 level. The coefficient for "Percent African 
American" has turned from positive to negative, and the coefficient for 
"Percent Hispanic / Latino" has become strongly negative. It seems likely 
that these two factors are acting as proxies for cultural and economic 
factors with more predictive power. 

Residuals from the five-factor model of Table 2-6 are shown in Table 2-8. 
This include standard error estimates based on results from the trends 
study of Section 2A. 



Table 2-8 

Residuals for MCAS Grade 10 Math Test Scores, 5-Factor Model 



City or Town 


High School 


Residual 


Std. Error 


Ratio 


Arlington 


Arlington 


4.6 


2.6 


1.8 


Belmont 


Belmont 


12.2 


2.7 


4.5 


Boston 


Boston High 


-8.9 


5.1 


-1.7 


Boston 


Brighton 


1.7 


3.3 


0.5 


Boston 


Charlestown 


1.0 


4.8 


0.2 


Boston 


Dorchester 


-1.6 


4.3 


-0.4 


Boston 


East Boston 


2.1 


4.2 


0.5 


Boston 


Hyde Park 


-6.1 


4.8 


-1.3 


Boston 


Jeremiah Burke 


4.1 


5.0 


0.8 


Boston 


South Boston 


-8.6 


3.7 


-2.3 


Boston 


The English High 


11.4 


5.0 


2.3 


Boston 


West Roxbury 


1.2 


3.9 


0.3 


Boston Exam 


Boston Latin 


21.3 


3.5 


6.0 


Boston Exam 


Latin Academy 


4.0 


4.1 


1.0 


Boston Exam 


O'Bryant Science 


4.65 


4.3 


1.1 


Braintree 


Braintree 


-1.78 


2.5 


-0.7 


Brookline 


Brookline 


9.5 


2.3 


4.1 


Cambridge 


Rindge & Latin* 


-6.3 


4.0 


-1.6 


Chelsea 


Chelsea 


2.2 


7.0 


0.3 


Dedham 


Dedham 


-1.6 


3.1 


-0.5 


Everett 


Everett* 


-2.8 


2.6 


-1.1 


Lexington 


Lexington 


4.4 


2.8 


1.6 


Lynn 


Classical 


-6.5 


3.3 


-2.0 


Lynn 


English 


-5.0 


3.9 


-1.3 


Malden 


Malden 


-6.6 


3.2 


-2.1 


Marblehead 


Marblehead 


3.4 


3.2 


1.1 


Medford 


Medford* 


-7.4 


2.7 


-2.7 


Melrose 


Melrose 


-2.7 


2.7 


-1.0 


Milton 


Milton 


-1.7 


3.1 


-0.6 


Newton 


North* 


8.6 


2.3 


3.7 


Newton 


South 


10.0 


2.8 


3.6 
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Peabody 


Veterans* 


-7.2 


2.5 


-2.9 


Quincy 


North Quincy 


-9.2 


3.6 


-2.5 


Quincy 


Quincy* 


-14.1 


3.1 


-4.5 


Revere 


Revere* 


-10.3 


2.7 


-3.9 


Salem 


Salem* 


-1.5 


3.1 


-0.5 


Saugus 


Saugus 


-2.8 


2.9 


-1.0 


Somerville 


Somerville* 


2.7 


4.7 


0.6 


Stoneham 


Stoneham 


-1.7 


3.1 


-0.5 


Swampscott 


Swampscott 


11.8 


3.2 


3.7 


Wakefield 


Memorial 


-2.0 


2.8 


-0.7 


Waltham 


Waltham* 


-7.4 


2.6 


-2.8 


Watertown 


Watertown 


4.7 


3.1 


1.5 


Weymouth 


Weymouth* 


-7.0 


2.3 


-3.0 


Winchester 


Winchester 


12.5 


2.9 


4.4 


Winthrop 


Winthrop* 


-5.5 


3.5 


-1.6 


Woburn 


Woburn 


-1.9 


2.6 


-0.7 



* school providing vocational education 



Sources of data: Appendix 3, Appendix 4, Statistica model 

At first glance, some residuals in Table 2-8 look substantial, several scale 
points of difference from the average scores predicted by the model. 
However, residual ratios for most schools are within +/- 2 standard errors, 
not significant at a p<.05 level. Someone familiar with metropolitan 
Boston will recognize that schools with high and low residual ratios tend 
to be in high-income and low-income communities, respectively. It 
therefore seems likely that adding a factor for incomes can increase the 
predictive power of the model. 

The most recent community income data were from the US Census of 
1990, for 1989 per-capita income. Comparable 1999 income statistics 
were not yet available. The Massachusetts Department of Revenue could 
produce current community income statistics but has not done so; the state 
continues to use 1989 federal census data on incomes to apportion aid to 
public schools. After adding 1989 per-capita community income in 
$ 1,000s as a factor (Mass. DoR, 1999), without any attempt to adjust 
incomes so as to reflect school districts or student households, the model 
associates 80 percent of the statistical variance, and factor weights became 
those shown in Table 2-9. 



Table 2-9 

6-Factor Model for 1999 MCAS Grade 10 Math Test Scores 



Factor 


Coefficient 


Standard Error 


Intercept, for all factors zero 


202.7 


5.400 


B. Percent African American 


-0.020 


0.083 


C. Percent Asian or Pacific Islander* 


0.371 


0.121 


D. Percent Hispanic / Latino 


0.044 


0.144 


E. Percent limited English proficiency* 


-0.695 


0.171 
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F. Percent free or reduced price lunch 


0.050 


0.131 


H. Per-capita community income (1989)* 


1.186 


0.230 



Sources ot data; Appendix 3, Appendix 4, Statistica model 



Three factors in Table 2-9 (marked *) have substantial significance, at a 
p<.005 level or better, and three have very low significance. Factor weight 
has shifted from "Percent free or reduced price lunch" to "Per-capita 
community income (1989)," while "Percent limited English proficiency" 
retains a large coefficient and high significance. Dropping low- 
significance factors, the resulting three-factor model is shown in Table 2- 
10 . 



Table 2-10 

3-Factor Model for 1999 MCAS Grade 10 Math Test Scores 



Factor 


Coefficient 


Standard Error 


Intercept, for all factors zero 


204.9 


4.446 


C. Percent Asian or Pacific Islander 


0.381 


0.109 


E. Percent limited English proficiency 


-0.626 


0.081 


H. Per-capita community income (1989) 


1.104 


0.197 



Sources ot data: Appendix 3, Appendix 4, Statikica model 



The three-factor model of Table 2-10 also associates 80 percent of the 
statistical variance. All of its factors are statistically significant at a p<.001 
level. 

For each school included in these studies. Table 2-11 presents adjusted 
average 1999 tenth-grade MCAS mathematics test scores and residuals 
from the three-factor statistical model of Table 2-10, with the uncertainties 
in average scores and residuals expressed as standard errors, based on the 
variance estimate calculated in the trends study of Section 2A. 



Table 2-11 

Residuals for MCAS Grade 10 Math Test Scores, 3-Factor Model 



City or 
Town 


High School 


Average 


Std. 

Error 


Residual 


Std. 

Error 


Arlington 


Arlington 


234 


2.1 


4.2 


2.4 


Belmont 


Belmont 


243 


2.2 


6.7 


2.7 


Boston 


Boston High 


204 


2.9 


-10.3 


3.1 


Boston 


Brighton 


205 


2.2 


-0.1 


2.9 


Boston 


Charlestown 


206 


2.7 


-1.6 


3.7 


Boston 


Dorchester 


204 


3.0 


-0.5 


3.5 


Boston 


East Boston 


205 


2.3 


-1.0 


2.9 


Boston 


Hyde Park 


203 


3.3 


-5.0 


3.7 


Boston 


Jeremiah Burke 


208 


2.7 


5.1 


3.4 


Boston 


South Boston 


205 


2.6 


-7.8 


3.1 


Boston 


The English 
High 


204 


2.3 


9.7 


3.7 


Boston 


West Roxbury 


205 


2.0 


0.0 


2.7 



httn ://eDaa.asu.edu/ enaa/v9n42/ 






10/30/01 






EPAA Vol. 9 No. 4...: Significance of Test-based Ratings for Metropolitan Boston School Page 20 of 60 



Boston Exam 



Boston Exam 



Boston Exam 



Braintree 



Brookline 



Cambridge 



Chelsea 



Dedham 



Everett 



Lexington 



Lynn 



Lynn 



Malden 



Marblehead 



Medford 



Melrose 



Milton 



Newton 



Newton 



Peabody 



Quincy 



Quincy 



Revere 



Salem 



Saugus 



Somerville 



Stoneham 



Swampscott 



Wakefield 



Waltham 



Watertown 



Weymouth 



Winchester 



Winthrop 



Woburn 



Boston Latin 



Latin Academy 



O'Bryant 

Science 



Braintree 



Brookline 



Rindge & 
Latin* 



Chelsea 



Dedham 



Everett* 



Lexington 



Classical 



English 



Malden 



Marblehead 



Medford* 



Melrose 



Milton 



North* 



South 



Veterans* 



North Quincy 



Quincy' 



Revere* 



Salem* 



Saugus 



Somerville* 



Stoneham 



Swampscott 



Memorial 



Waltham* 



Watertown 



Weymouth* 



Winchester 



Winthrop’ 



Woburn 



254 



233 



227 



228 



240 



220 



216 



227 



221 



238 



216 



213 



221 



232 



221 



226 



228 



239 



242 



220 



227 



212 



218 



220 



226 



216 



227 



240 



227 



220 



231 



222 



243 



223 



226 



1.7 



2.1 



2.1 



1.9 



1.6 



1.6 



2.3 



2.5 



1.9 



1.7 



2.0 



2.1 



2.0 



2.7 



2.2 



2.1 



2.3 



1.5 



2.0 



1.8 



1.9 



2.3 



1.9 



2.1 



2.2 



1.8 



2.5 



2.6 



2.2 



1.8 



2.6 



1.5 



2.3 



3.0 



2'.0 



23.6 



5.0 



4.3 



1.7 



-0.5 



-4.9 



3.7 



1.3 



2.3 



■5.8 



•3.3 



0.9 



• 2.8 



■6.3 



■2.3 



1.2 



■ 2.0 



0.8 



2.2 



■2.9 



■7.2 



11.9 



■ 6.1 



0.7 



0.8 



1.7 



1.5 



7.6 



0.6 



-4.0 



4.3 



-3.8 



2.8 



-1.4 



1.3 



2.7, 



2.8 



3.4 



2.3 



2.7 



1.8 



2.8 



2.8 



2.5 



3.0 



2.8 



2.5 



2.6 



3.6 



2.5 



2.5 



2.6 



2.5 



2.8 



2.3 



3.0 



2.6 



2.4 



2.5 



2.6 



2.1 



2.9 



3.0 



2.6 



2.1 



2.8 



2.1 



3.2 



3.3 



2.4 



* school providing vocational education 
Sources of data: Appendix 3, Appendix 4, Statistica model 



Stepwise analysis shows that combinations of the factors in the three- 
factor model of Table 2-10 associate statistical variance in the amounts 
listed in Table 2-12. 



httni/Zenaa. asu.edu/enaa/v9n42/ 






10/30/01 




EPAA Vol. 9 No. 4...: Significance of Test-based Ratings for Metropolitan Boston School Page 21 of 60 



Table 2-12 

Factor Combinations for 1999 MCAS 
Grade 10 Math Test Scores 



Factors 




C 


.01 


E 


.62 


H 


.47 


CE 


.65 


CH 


.52 


EH 


.74 


CEH 


.80 



Islander 

E. Percent limited English proficiency 
H. Per-capita community income (1989) 

Sources of data: Appendix 3, Appendix 4, Statistica model 



Table 2-12 shows that the major factors are "Percent limited English 
proficiency" and "Per-capita community income (1989)." Although 
statistically significant, "Percent Asian or Pacific Islander" is a weak 
cofactor, associating only 1 percent of the variance by itself. 

The three-factor model of Table 2-10 was evaluated for predictions of 
1998 and 2000 average test scores. It was not expected to perform as well, 
since most factor data were for 1999. However, the factor weights and 
significance proved robust, and the model associated statistical variance as 
shown in Table 2-13. 



Table 2-13 

Year Comparisons for MCAS 
Grade 10 Math Test Scores 



Year 




1998 


.83 


1999 


.80 


2000 


.77 



Sources of data: Appendix 3, Appendix 4, Statistica model 



The residuals in Table 2-1 1 suggest systematic contributions to test score 
averages at some schools that might have been produced by unusual 
efforts. Probable outliers for the predictive model with correspondingly 
large year-to-year average score changes had a strongly positive bias: 
those for Boston Latin, Latin Academy and Swampscott. 

Model behavior for metropolitan Boston schools may be biased by special 
characteristics of City of Boston schools, because the students who score 
well on school-based standard tests are selected for admission to the three 
exam schools. (Note 9) Current data also attribute average Boston per- 
capita income equally to all school districts instead of adjusting by 
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districts or census tracts. The Boston cross-enrollment and busing 
programs would complicate an income analysis. Behavior of the three- 
factor model of Table 2-10 was examined for 1999 tenth-grade MCAS 
mathematics test scores, considering only schools outside the City of 
Boston. These 34 schools had a 1 999 total population per grade of about 
10,200 students out of 13,730 for all schools considered in these studies. 

When applied only to schools outside the City of Boston, the three-factor 
model of Table 2-10 showed significance at the p<.05 level for "Percent 
limited English proficiency" and "Per-capita community income (1989)" 
but not for "Percent Asian or Pacific Islander." A two-factor model based 
on the first two of these factors, with schools weighted by numbers of test 
participants, associates 86 percent of the statistical variance. Factor 
weights became as shown in Table 2-14. 



Table 2-14 

2-Factor Model for 1999 MCAS Grade 10 Math Test Scores 



Factor 


Coefficient 


Standard Error 


Intercept, for all factors zero 


201.5 


2.934 


E. Percent limited English proficiency 


-0.325 


0.136 


H. Per-capita community income (1989) 


1.307 


0.126 



Sources ot data: Appendix 3, Appendix 4, Statistica model 



Several trial factors were added individually to the two-factor model of 
Table 2-14, but none showed statistical significance at the /><.05 level. The 
statistical variance associated when each trial factor was added to this two- 
factor model is shown in Table 2-15. 



Table 2-15 

Factor Comparison for 1999 MCAS Grade 10 Math Test Scores 





Trial Factor Added 


.86 


None 


.88 


A. Population per grade 


.88 


B. Percent African American 


.86 


C. Percent Asian or Pacific Islander 


.86 


D. Percent Hispanic / Latino 


.87 


F. Percent free or reduced price lunch 


.87 


G. Percent reduction, grades 9+10 to 11+12 


.87 


Percent special education 


.86 


Per-capita property value, 1998, $000s 


.88 


Spending, regular education, $000s 



Sources ot data: Appendix J, Appendix 4, Statistica model 



The last three trial factors in Table 2-15 are districtaverages; the last and 
third from last are for all schools in all grades. Three districts, Lynn, 
Newton and Quincy, each operate two academic high schools which will 
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not be distinguished by these factors. Outside the City of Boston, only 
"community income" and "limited English proficiency" are significant 
contributors to 1999 tenth-grade MCAS mathematics test scores; their 
indicators are effective predictors. 

A one-factor model, using only "Per-capita community income (1989)," 
performed almost as well as any combination of factors shown in Table 2- 

15, associating 84 percent of the variance. The factor weight is in Table 2- 

16. 



Table 2-16 

1-Factor Model for 1999 MCAS Grade 10 Math Test Scores 



Factor 


Coefficient 


Standard Error 


Intercept, for all factors zero 


197.0 


2.395 


H. Per-capita community income (1989) 


1.465 


0.114 



Sources ot data: Appendix J, Appendix 4, Statikica model 



"Percent limited English proficiency" is much less effective in a one-factor 
model, associating only 38 percent of the variance. Community income 
appears to be the dominant factor associated with these test scores. The 
1999 adjusted average tenth-grade MCAS mathematics test scores by 
school, plus residuals from the two-factor and one-factor models for 1999, 
shown in Table 2-14 and Table 2-16, with standard error estimates for 
each, are in Table 2-17. 



Table 2-17 

Residuals for MCAS Grade 10 Math Test Scores, 1,2-Factor Models 



City or 
Town 


mp 

School 


Average 

Score 


Error 


2-factor 

Residual 


“Sta: 

Error 


1-factor 

Residual 


^ta: 

Error 


Arlington 


Arlington 




2.1 


4.7 


2.2 


5.6 


2.2 


Belmont 


Belmont 


7^ 


2.2 


6.6 


2.4 


6.8 


7 :a 


Braintree 


Braintree 


77S 


1.9 


2.3 


2.1 


3.7 


2.0 


Brookline 


Brookline 


7M 


1.6 


1.4 


2.0 


0.5 


2.0 


Cambridge 


Rindge& 

Latin* 


77^ 


1.6 


- 5:o 


1.8 


- 6.1 


1.8 


Chelsea 


Chelsea 


2l6 


2.3 


4.3 


2.7 


2.1 


2.6 


Dedham 


Dedham 


3T7 


2.5 


1.1 


2.6 


2.1 


2.6 


Everett 


Everett* 


7H 


1.9 


2.6 


2.1 


3.2 


2.2 


Lexington 


Lexington 


77% 


1.7 


-3.3 


2.2 


- 4.0 


2.2 


Lynn 


Classical 


216 


2.0 


2.2 


2.4 


-0.1 


2.2 


Lynn 


English 


7X7 


2.1 


1.0 


2.9 


-3.1 


2.4 


Malden 


Malden 


2IT 


2.0 


2.2 


2.2 


0.9 


2.1 


Marblehead 


Marblehead 


7X7 


2.7 


-9.2 


3.0 


-9.8 


3.1 


Medford 


Medford* 


7XX 


2.2 


- 1.7 


2.3 


-0.8 


2.3 


Melrose 


Melrose 


7XZ 


2.1 


- 1.7 


2.3 


-0.6 


2.2 


Milton 


Milton 


7XX 


2.3 


-2.6 


2.4 


-1.9 


2.4 


Newton 


North* 


7X^ 


1.5 


0.6 


1.9 


-0.2 


1.9 


Newton 


South 


7X1 


2.0 


3.4 


2.3 


2.8 


2.3 


Peabody 


Veterans* 


22U 


1.8 


- 3.1 


2.0 


- 1.9 


1.9 


Quincy 


North 


— 7n 


1.9 


2.7 


2.2 


4.5 


2.0 
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i^uincy 














Quincy 


Quincy’^ 


7n 


2.3 


-9.4 


2.5 


^TTJ3 


2.5 


Revere 


Revere* 




1.9 


-1.3 


2.1 


-0.5 


2.1 


Salem 


Salem* 


7K5 


2.1 


-0.4 


2.3 


-0.6 


2.3 


Saugus 


Saugus 


71^ 


2.2 


1.2 


2.4 


3.0 


2.3 


Somerville 


Somerville* 


TTE 


1.8 


-0.4 


2.3 


-3.2 


2.0 


Stoneham 


Stoneham 


TTf 


2.5 


1.7 


2.7 


3.3 


2.6 


Swampscott 


Swampscott 


7^ 


2.6 


5.5 


2.7 


5.6 


2.8 


Wakefield 


Memorial 


777 


2.2 


0.7 


2.4 


2.2 


2.3 


Waltham 


Waltham* 


77U 


1.8 


-2.3 


2.0 


-1.6 


2.0 


Watertown 


Watertown 


77T 


2.6 


3.8 


2.7 


4.2 


2.7 


Weymouth 


Weymouth’*' 


771 


1.5 


-3.5 


TJ 


-1.9 


1.7 


Winchester 


Winchester 


7^ 


2.3 


1.5 


~T7 


1.2 


2.7 


Winthrop 


Winthrop* 


777 


3.0 


-1.3 


3.2 


-0.1 


3.1 


Woburn 


Woburn 


776 


2.0 


1.3 


2.1 


2.4 


2.1 



* school providing vocational education 
Sources of data: Appendix 3, Appendix 4, Statistica model 



In an attempt to improve accuracy of the model in Table 2-14, schools 
with residuals from the two-factor model for 1999 that were greater than 
two standard deviations were dropped, Belmont with positive residual and 
Cambridge Rindge & Latin, Marblehead High and Quincy High with 
negative residuals. The two-factor model for 1 999 scores then produced 
the factor weights shown in Table 2-18. 



Table 2-18 

2-Factor Trial for 1999 MCAS Grade 10 Math Test Scores 



Factor 


Coefficient 


Standard Error 


Intercept, for all factors zero 


200.6 


2.188 


E. Percent limited English proficiency 


-0.216 


0.101 


H. Per-capita community income (1989) 


1.357 


0.095 



Sources ot data: Appendix 3, Appendix 4, Statikica model 



Chi square for the two-factor model of Table 2-18 was 36.2 with 27 degrees of 
freedom (p=.l 1). The one-factor model of Table 2-16 was also evaluated for 1999 
with the set of cases used in Table 2-18, producing the factor weights shown in 
Table 2-19. 



Table 2-19 

1-Factor Trial for 1999 MCAS Grade 10 Math Test Scores 



Factor 


Coefficient 


Standard Error 


Intercept, for all factors zero 


197.6 


1.793 


H. Per-capita community income (1989) 


1.463 


0.087 



Sources of data: Appendix 3, Appendix 4, Statistica model 



Chi square for the one-factor model of Table 2-19 was 43.4 with 28 degrees of 
freedom (p=.03). However, the cases and models from Table 2-18 and Table 2-19 
did not provide significant chi square probabilities for 1998 or 2000 scores; all 
other attempts to improve estimation by removing outliers also proved unstable. 
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Residuals from the two-factor model of Table 2-14 for schools outside the City of 
Boston are strongly autocorrelated. The coefficient was .45 between 1998 and 
1999 and .67 between 1999 and 2000. Scatterplots for successive years are shown 
in Figure 2-4 and Figure 2-5, comparing 1998 and 2000 residuals with 1999 
residuals: 

Figure 2-4: MCAS Grade 10 Math Test Score Residuals, 1998 versus 1999 



Residuals, 2-factor weighted model, 1 998 vs. 1 999 




1999 Residual 



Figure 2-5: MCAS Grade 10 Math Test Score Residuals, 2000 versus 1999 
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-8 



■io 

o 



-4 



0 4 

1999 Residual 



8 



12 



To the extent that a trend can be observed by applying the model of Table 2-14 to 
different years, it appears that schools scoring higher than predicted tend to 
increase scores in successive years, and schools scoring lower than predicted tend 
to decrease scores. Departures from model predictions are not all random. The 
two spans of years available for analysis suggest a systematic trend that could 
stratify high-scoring and low-scoring schools. 

C. Observations 

The trends and effects studies presented in Section 2A and Section 2B show how 
studies of these types tend to yield results with low statistical significance unless 
"something unusual" is going on. In the cases of the 2000 tenth-grade 
mathematics test and of the high scores and large increases at Boston Latin, Latin 
Academy and Swampscott, the studies do indicate "something unusual," although 
they cannot tell what it is. They also illustrate that commonly published "league 
tables" of scores strongly reflect social factors associated with school populations, 
not factors clearly associated with school performance. The effects study shows a 
robust, positive correlation of scores with household incomes, plus a smaller, 
negative correlation with limited English proficiency. Factor weights and 
statistical significance for other factors considered are small. School-based data 
are not currently published for several potentially significant factors, such as class 
sizes in elementary grades, mathematics course enrollments and levels of teacher 
preparation. 

By far the strongest factor in predicting tenth-grade MCAS mathematics test 
scores is "Per-capita community income (1989)." For the schools outside the City 
of Boston this factor alone performed nearly as well as all available factors 
combined, associating 84 percent of the variance compared with 88 percent when 
all available factors were used. The scatterplot in Figure 2-6 shows 1999 average 
scores for these schools versus 1989 per-capita community income: 

Figure 2-6: 1999 MCAS Grade 10 Math Test Scores versus Community 

Income 



Schools Inside Route 1 28, Except Boston 
Predicted Score = 1 97.4 + 1 .45 Income (1 989, $1 000s) 
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Per-Capita Income, 1 989, $1 ,000s 

In Figure 2-6, the relation between school-averaged test scores and per-capita 
community income looks linear over an income range of about 2‘/2 to 1 for this set 
of schools. There is no obvious threshold or saturation behavior. As it happens, 
spending on regular education programs in school districts also varies over a 
range of about 2 'A to 1 for those schools (Mass. DoE, 2000f). However, there is 
only a weak relationship between 1999 spending on regular education programs 
in those school districts emd their 1999 tenth-grade MCAS mathematics test 
scores, associating only about 3 percent of the variance and not statistically 
significant at a p<.05 level, as shown in the scatterplot of Figure 2-7. 

Figure 2-7: 1999 MCAS Grade 10 Math Test Scores versus School Spending 



Schools Inside Route 1 28, Except Boston 




Per-Student School District Spending, Regular Education, 1 999, $1 ,000s 



Although a tendency for scores on school-based standard tests to rise with 
incomes has been recognized in the US for more than 70 years (Bolon, 2000), 
there has been relatively little research on this phenomenon in ordinary income 
ranges, compared with the attention given to associations between test scores and 
conditions of poverty or race. The strong, apparently linear association of average 
test scores with community incomes shown here, as contrasted with their weak 
association with school spending, calls for investigation but is beyond the scope 
of this report. It seems likely that community incomes are providing something 
beyond what school programs provide, but if so we cannot tell from these data 
what it might be. Perhaps this effect should not be surprising, since most students 
spend more than three-fourths their waking hours outside school. 

The factor "Percent limited English proficiency" was the second strongest 
influence on predicted test scores. (Note 10) Previous effects studies of MCAS 
scores might not have shown this if they failed to utilize the entire variety of 
school-associated data available, including "limited English proficiency," "racial" 



httn://enaa. asii.edu/enaa/v9n47/ 



398 



10/30/01 





EPAA Vol. 9 No. 4...: Significance of Test-based Ratings for Metropolitan Boston School Page 28 of 60 



or "ethnic," and "free or reduced price lunch" student categories. A hypothesis 
inviting study is that students classified with "limited English proficiency" might 
receive mathematics instruction less relevant to curriculum tested by MCAS than 
other students. Another hypothesis is that strengthening English language 
instruction for "limited English proficient" students might improve their MCAS 
mathematics test scores, provided all current efforts to teach mathematics and 
other subjects are maintained. While statistical associations suggest these 
conjectures, only observations and experiments could prove or disprove them. 

Factors of "Percent African American" and "Percent Hispanic / Latino" did not 
make significant contributions to school-averaged tenth-grade MCAS 
mathematics test scores after other factors were introduced. Although "Percent 
Asian or Pacific Islander" retains statistical significance in certain models for 
some years, it is a much weaker factor than "Per-capita community income 
(1989)" or "limited English proficiency." The latter factors, and not "racial" or 
"ethnic" percentages, provide by far the strongest statistical associations with 
school-averaged tenth-grade MCAS mathematics test scores in metropolitan 
Boston. (Note 1 1) 

Statistical significance of test-based ratings has been the principal focus of the 
studies. Some studies developing or using such ratings do not provide an analysis 
of variability, without which significance cannot be determined; or they estimate 
variability from single test session reliability measurements, which were shown to 
be optimistic (for example, Gaudet, 2001, and Tuerck, 2001a). The trends study 
in Section 2A of this report provides evidence derived from tenth-grade MCAS 
mathematics test scores for larger variability estimates. 

D. Summary Analysis 

My summary analysis is based on a one-factor model for metropolitan Boston 
communities that each operate only a single academic high school, weighted by 
numbers of students tested. The effects study showed that "Per-capita community 
income (1989)" was the dominant factor in predicting 1999 school-averaged 
tenth-grade MCAS mathematics test scores. All other factors made only small 
contributions to predictions with much lower significance. From published data, 
community income could not be estimated reliably for each of the multiple 
academic high schools in Boston, Lynn, Newton and Quincy. Reduction in scope 
leaves 28 high schools in the same number of communities, with a total of about 
8,200 students per grade recorded for 1999. Estimates of uncertainties in school- 
averaged test scores are based on findings of the trends study, which showed 
year-to-year variability of school-averaged scores several times greater than the 
variability implied by conventional test reliability measurements. Plots of results 
include uncertainty intervals (sometimes called "error bars") equivalent to +!- 1.4 
estimated standard errors. When item intervals do not overlap, when standard 
errors have been accurately estimated, and when items are uncorrelated, then 
differences between items are significant at the p<.05 level. The one-factor model 
for 1999 with this set of cases associated 82 percent of the variance and produced 
the factor weights shown in Table 2-20. 



Table 2-20 

Community Income Model for 1999 MCAS Grade 10 Math Test Scores 



Factor 


Coefficient 


Standard Error 


Intercept, for all factors zero 


198.4 


2.667 


Per-capita community income (1989) 


1.397 


0.129 
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Sources of data; Appendix 3, Appendix 4, Statistica model 

The 1999 adjusted average tenth-grade MCAS mathematics test scores by school, 
plus residuals from the foregoing one-factor model for 1999, with standard error 
estimates for each, are shown in Table 2-21. 

Table 2-21 

Residuals for MCAS Grade 10 Math Test Scores, Income Model 



City or 
Town 


Average 

Score 


Std. 

Error 


1-factor 

Residual 


Std. 

Error 


Arlington 


234 


2.1 


5.6 


2.2 


Belmont 


243 


2.2 


7.1 


2.5 


Braintree 


228 


1.9 


3.5 


2.0 


Brookline 


240 


1.6 


1.0 


2.1 


Cambridge* 


220 


1.6 


-6.2 


1.8 


Chelsea 


216 


2.3 


1.4 


2.6 


Dedham 


227 


2.5 


2.0 


2.6 


Everett* 


221 


1.9 


2.7 


2.2 


Lexington 


238 


1.7 


-3.4 


2.3 


Malden 


221 


2.0 


0.5 


2.2 


Marblehead 


232 


2.7 


-9.2 


3.2 


Medford* 


221 


2.2 


-1.1 


2.3 


Melrose 


226 


2.1 


-0.7 


2.2 


Milton 


228 


2.3 


-1.8 


2.4 


Peabody* 


220 


1.8 


-2.2 


1.9 


Revere* 


218 


1.9 


-1.0 


2.1 


Salem* 


220 


2.1 


-1.0 


2.3 


Saugus 


226 


2.2 


2.7 


2.3 


Somerville* 


216 


1.8 


-3.6 


2.0 


Stoneham 


227 


2.5 


3.1 


2.7 


Swampscott 


240 


2.6 


5.8 


2.8 


Wakefield 


227 


2.2 


2.0 


2.3 


Waltham* 


220 


1.8 


-1.9 


2.0 


Watertown 


231 


2.6 


4.1 


2.7 


Weymouth* 


222 


1.5 


-2.1 


1.7 


Winchester 


243 


2.3 


1.8 


2.8 


Winthrop* 


223 


3.0 


-0.4 


3.1 


Woburn 


226 


2.0 


2.2 


2.1 



* school providing vocational education 



Sources of data; Appendix 3, Appendix 4, Statistica model 

The plot in Figure 2-8 shows school-averaged adjusted scores on the 1999 tenth- 
grade MCAS mathematics test and the corresponding uncertainty intervals (or 
"error bars"). In this and the next two plots, the 28 schools considered in this 
summary analysis have been rank-ordered from the lowest to the highest average 
scores on the 1999 tenth-grade MCAS mathematics test. 
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Figure 2-8: Average 1999 MCAS Grade 10 Math Test Scores by School 



Communities Inside Route 1 28 with One High School 




The plot in Figure 2-9 shows residuals for school-averaged 1999 tenth-grade 
MCAS mathematics test scores, the differences left after subtracting away 
predictions calculated from the factor "Per-capita community income (1989)." 

Figure 2-9: Residuals of 1999 MCAS Grade 10 Math Test Scores by School 



Communities Inside Route 1 28 with One High School 




The picture in the plot of average scores of Figure 2-8, with significant 
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separations between high-scoring and low-scoring schools, is shown by the 
residuals plot of Figure 2-9 to be associated largely with differences in 
community income. Chi square for the residuals distribution is 63.2 with 27 
degrees of freedom. After subtracting predictions based on community income, 
residuals of average scores settle to just a little more than statistical noise. Only 
five or six schools can be reliably distinguished other than by community income. 

The last plot. Figure 2-10, shows the changes in school-averaged 1999 tenth- 
grade MCAS mathematics test scores from the same schools' average scores in 
1998. 



Figure 2-10: Changes in MCAS Grade 10 Math Test Scores by School, 1998- 

1999 



Communities Inside Route 128 with One High School 




There was low statistical significance in year-to-year changes of the school- 
averaged adjusted scores on tenth-grade MCAS mathematics tests between 1998 
and 1999. The distribution of changes about an average change of +1.2 scale 
points, as shown in Figure 2-10, fits a normal distribution with weighted chi 
square of 22.7 for 26 degrees of freedom ip = .65). Despite the possibility of little 
significance in score changes, such changes are criteria by which the 
Massachusetts Department of Education has begun to rate school Performance 
(Note 12) as "Failed to Meet Expectations," "Approached Expectations," "Met 
Expectations" or "Exceeded Expectations." There are sev.ere penalties, including 
state closure or seizure, for schools with low scores and ratings of "Failed to Meet 
Expectations." 

The statistical significance of differences in school-averaged scores, aside from 
their reflection of community income, may not be not enough to compare schools 
reliably and may not be enough to evaluate short-term changes in teaching and 
learning. Whatever aspects of school performance these scores might measure can 
be lost in the fluctuations for a particular school over any one year or few years. 
Only averages and trends in scores over several years would be likely to yield 
statistically useful information. 
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Some observers question whether it is realistic to expect standardized tests to 
yield significant information comparing school performance, even over a period 
of years (for example, Rowe, 1999a, and Rowe, et al., 1999b). They argue that 
variations in student achievement are commonly greater within schools than 
between schools. Others contend that adaptive and defensive behavior encouraged 
by "high-stakes" political environments grossly distorts outcomes from all types 
of educational assessment and robs them of meaning (for example, Sacks, 1999, 
and Hayman, 1998). It is important to point out that significance attributed to 
results in these studies is purely statistical. Neither these studies nor any others 
known to the author have shown that MCAS test scores have practical 
significance, in the sense of predicting success in adult activities to any greater 
degree than could be done with knowledge of student backgrounds. 

Section 3: Results 

A. Opportunities and Questions 

In future work, it may be helpful to examine categories of metropolitan Boston 
schools that were excluded from or specially identified in these studies: 

• Vocational, technical and agricultural schools 

• Academic schools that also provide vocational education 

• Pilot schools, charter schools, specialty schools and small schools 

Other questions might be answered by extending data coverage: 

• Do other test scores yield similar results? 

• Do small class sizes have significant effects? 

• Does teacher preparation have significant effects? 

• Do math course enrollments have significant effects? (Note 13) 

• Can other factors be found that increase significance? 

• Are the same effects found in other Massachusetts schools? 

• Are similar effects observed in elementary and middle school grades? 

• Are the same or different patterns observed in other US metropolitan areas? 

• Can income factors be estimated for communities with multiple high 
schools? 

• If individual data were available, would multi-level analysis show different 
results? 

The strong, linear relation found between school-averaged tenth-grade MCAS 
mathematics test scores in a relatively calm year and community income in a 
previous year, within ordinary ranges of incomes, leads to several questions. Does 
community income primarily determine educational achievement, regardless of 
school performance? Do MCAS and similar tests measure skills and knowledge 
that are acquired in schools, or do they measure skills and knowledge that eire 
largely acquired outside schools? Do communities with substantially different 
incomes have substantially different expectations for student performance on 
MCAS and similar tests? Is current community income also strongly correlated 
with test scores? After a turbulent period of local efforts to raise scores, will the 
correlation between income and scores remain as strong? Exploring and 
understanding some of these issues will require a different approach. 

B. Conclusions 
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Community income has been found strongly correlated with tenth-grade MCAS 
mathematics test scores and associated more than 80 percent of the variance in 
school-averaged 1999 scores for a sample of Boston-area communities. The 
influence of community income was robust against several sets of model 
variables and cases. 

Community income swamped the influence of the other social and school factors 
examined. Once community income was included in models, other factors— 
including percentages of students in disadvantaged populations, percentages 
receiving special education, percentages eligible for free or reduced price lunch, 
percentages with limited English proficiency, school sizes, school spending 
levels, and property values--all failed to associate substantial additional variance. 

Large uncertainties in residuals of school-averaged scores, after subtracting 
predictions based on community income, tend to make the scores ineffective for 
rating performance of schools. Large uncertainties in year-to-year score changes 
tend to make the score changes ineffective for measuring performance trends. In 
their present state, considered as a means to rate the performance of public 
schools, tenth-grade MCAS mathematics tests mainly appear to provide a 
complex and expensive way to estimate community income. 

Appendix 1: Education Reform in Massachusetts 

Massachusetts has experienced many education experiments and reforms over the 
past few centuries. (Note 14) During the 1990s Massachusetts education reform 
was driven by the McDuffy school finance lawsuit (McDuffy, 1993), originally 
filed in 1978. In its McDuffy decision, the Massachusetts Supreme Judicial Court 
said that Massachusetts funding disparities harmed the quality of education for 
some students, denying them education to which they were constitutionally 
entitled. This June, 1993, decision was widely anticipated. The Massachusetts 
Education Reform Act of 1993 (Education Reform, 1993) was signed less than a 
week after the decision was released. 

A group called the Massachusetts Business Alliance for Education, (Note 15) 
organized in 1988 and led by the late John C. (Jack) Rennie, then CEO of the 
former Pacer Infotec, Inc., of Burlington, MA, (now the AverStar division of 
Titan Corp., San Diego, CA), and S. Paul Reville, then director of the Worcester 
Public Education Fund, wrote the reform bill sponsored by the Education 
Committee of the legislature. In 1991 the Business Alliance produced a document 
entitled Every Child a Winner. (Note 16) A story from the May 2, 1993, 
Northwest edition of the Boston Globe quoted former Rep. Mark Roosevelt, then 
House Education Committee Chair, as saying that the House education reform 
bill then pending "is essentially [the Business Alliance document]." In a 
publication of MassINC, Rennie is quoted as saying, "We bought 
change" (Walser, 1997). Most of this work was carried out in secret. As late as 
December, 1992, then Lt. Gov. Cellucci was calling on the Education Committee 
chairs. Sen. Thomas Birmingham of Chelsea and Rep. Roosevelt of Beacon Hill, 
to disclose their bill (Howe, 1992). Almost all the controversy generated by this 
legislation focused on its funding formulas. Until 1993, the public had hardly £iny 
knowledge of its sweeping changes in school policy and regulation. The 
following newspaper report was printed December 23, 1992 (Overdue, 1992): 

"The bill also calls for higher student achievement and curriculum 

standards." 
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This was the most thorough description in mainstream news media from 1988 
through 1992. The bill was released in an emergency legislative session of 
January 4-5, 1993, but it failed to pass. 

Soon after the bill became public, education and public interest groups began to 
react. As reported in the Boston Globe on January 26, 1993, a coalition headed by 
Stephen Bing of the Massachusetts Advocacy Center predicted major problems 
with the legislation, including these (Ribadeneira, 1993): 

• The reform bill will institutionalize unfair teaching practices such as using 
tests to track students into different ability levels. 

• By substituting different certificates in place of the high school diploma, 
the bill will contribute to the dropout problem rather than ameliorate it. 

• The legislation provides no mechanism for meaningful participation by 
parents or students in the development of a remedial education plan nor any 
opportunity to contest an inadequate plan. 

Such objections were ignored. Neither the mainstream news media nor the Great 
and General Court gave these or other educationally oriented issues further 
attention in 1993. 

Rep. Thomas Finneran of Mattapan, then chair of the House Ways and Means 
Committee, secretly inserted anti-abortion provisions in the bill, provoking a 
storm of protest (Howe, 1993a). Other House controversy centered on a salary 
cap for teachers, which was removed. Proposals for "school choice," charter 
schools and gambling revenues became the focus of activity in the Senate. The 
bill quickly became a hodge-podge of added provisions with no coordination. 
Many observers became skeptical about overall benefit. Geoffrey Beckwith of the 
Massachusetts Municipal Association was quoted as saying, "It certainly doesn't 
appear at this time that this bill will bring about any fundamental reform" (Howe, 
1993b). In February, 1993, the Supreme Judicial Court heard testimony in the 
McDuffy case. In March, former Rep. Roosevelt began a (losing) campaign for 
Governor (Lehigh, 1993, Howe, 1993c). In April, the Edison Project, a business 
corporation, announced interest in privatizing Massachusetts schools (Nealon, 
1993). By May, an impasse over "school choice" had developed, the then Senate 
President William Bulger of Boston demanding it and the then House Speaker 
Charles Flaherty of Cambridge rejecting it. At the time, the Business Alliance 
opposed the "school choice" and charter school amendments (Taylor, 1993). 
However, another business group calling itself "CEOs for Fundamental Change in 
Education" had appeared, dominated by banking and large business interests and 
supported by the Pioneer Institute. It was actively promoting charter schools and 
"school choice" through the Massachusetts Senate (Vemiochi, 1993). After 
compromising with limits and delays on "school choice" and charter schools, the 
House passed the bill through second reading June 2 and the Senate passed it June 
3. The Supreme Judicial Court released its McDuffy case decision June 15. 
Former Gov. William Weld signed the Education Reform Act on June 18, 1993. 

In seven years under the Education Reform Act, state aid to Massachusetts public 
schools has grown from $1.3 billion to $3.0 billion per year, almost all the 
increase going to communities with low household incomes. For example, 
Holyoke, a low-income community, now receives over 90 percent of its school 
funding from the state, while Brookline, a high-income community, receives only 
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about 10 percent (Mass. DoE, 2000e). In 1992, Holyoke spent less than 75 
percent as much per student as Brookline, but now it spends about 95 percent of 
what Brookline does (School reform, 1992). Still, the Act has tended to provide 
more of a windfall for Holyoke's taxpayers than for its public school students. 

Besides setting state commitments to equalize school funding, the Education 
Reform Act made many changes to Massachusetts education policy and 
regulations, including the following, as described by the Business Alliance 
(Taylor, 1993): 

• New goals, standards and indicators of performance for schools, students 
and teachers 

• Financial rewards to teachers and schools that excel 

• Decentralized authority, limiting school committees to policy-making and 
oversight, making CEOs of superintendents, and giving hiring and firing 
power to principals 

• Preschool for all 3- and 4-year-olds 

• Expanded professional development for teachers 

More than seven years later, some of these changes are only starting to be 
implemented. 

Before 1996, the Massachusetts Board of Education regarded test scores as only 
one component of school accountability. In a 1993 policy advisory cited by 
Wheelock, 1999, the Board warned that an accountability system based primarily 
on test scores would be likely to produce harmful long-term consequences, 
including: exclusion of weaker students from the assessed pool of students; 
lowered morale among teachers and students; the loss of experienced educators 
from schools enrolling many disadvantaged students; distortion of instruction and 
curriculum to reflect test content and format; cheating and corruption of test 
scores. 

Nevertheless the Board began development of testing programs to satisfy the 
provisions of the Education Reform Act, which eventually became MCAS. It 
appointed committees of educators and parents to help insure that tests were 
meaningful, fair and free from overt forms of bias (French, 1998). From 1993 
through 1 996, Massachusetts invested more than $2 million to support education 
reform study groups seeking ways to set high expectations for students 
(Antonucci, 1997a). 

By March of 1996 the Department of Education had completed a Common Core 
of Learning (Mass. DoE, 1994), released six of seven planned curriculum 
frameworks based on it, and begun the development of MCAS based on the 
frameworks. In addition, it had announced plans (Mass. DoE, 1996a) to: award 
grants to school districts for assessment activities such as portfolio development; 
hold statewide conferences on local assessment strategies; publish examples of 
student work that meet the statewide standards so that districts have a model of 
what to strive for; develop a bank of assessment exercises linked to the 
curriculum frameworks for use by classroom teachers. All of these satisfy or 
support provisions of the Education Reform Act. 
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Development of MCAS took a sharp turn away from public participation (Mass. 
DoE, 1996b) after the appointment of John Silber as chair of the Massachusetts 
Board of Education in November, 1995 (Pawlack-Seaman, 1996). In August, 

1996, Silber, former president of Boston University and an unsuccessful 
candidate for Governor, working with then Gov. Weld, engineered replacement of 
the 17-member Board of Education, including four African-Americans and 
Latinos, with a 9-member board, including several with ties to school 
privatization and charter schools, only one African-American and no Latinos 
(Jackson, 1996, Wong, 1996). At his first meeting with the Board, Silber said the 
Education Reform Act's underlying principle that all students are capable of 
learning at high levels was "rubbish" (Future, 1996). Responding to a demand for 
his resignation in 1997 he commented, "Some of the things that pass for learning 
disabilities used to be called stupidity" (Pawlack-Seaman, 1997). Soon after the 
Board replacement, the committees of educators and parents that had been formed 
to oversee curriculum frameworks, test development and other education reforms 
were disbanded (Antonucci, 1997b). 

In December, 1996, Silber proposed a two-track system (Avenoso, 1996) with a 
general diploma awarded for passing the GED, a test introduced during World 
War II by Everett F. Lindquist, developer of the Iowa test series, and now 
administered by the American Council on Education. An honors diploma would 
be awarded for high scores on the Massachusetts test series. Silber was forced to 
abandon the plan in January, 1997, when his personally chosen Board of 
Education refused to support it (Leung, 1997). However, a legacy of Silber's 
proposal remains, the view that MCAS should be aimed at the exceptional 
student. In August, 1999, the Business Alliance revived the two-track concept 
(Still, 1999) with a proposal to award general diplomas to students who satisfy 
"essential requirements in English and math." The Business Alliance did not 
specify how this would be administered, and the Department of Education and 
Board of Education still oppose the concept. What they have done instead is to 
make a "competency determination" required by the Education Reform Act for a 
high-school diploma depend on achieving relatively low MCAS test scores, (Note 
17) answering about 40 percent of the questions. (Note 18) A "certificate of 
mastery," as specified by the Act, is to be awarded for much higher scores, (Note 

19) answering about 80 percent of the questions to achieve an "advanced" rating 
on one or more tests. 

After the loss of two Education Commissioners in rapid succession (Battenfield 
and Pressley, 1999), Silber resigned during a struggle over a new Commissioner 
in March, 1999. The outcome of the controversy (Estrin, 1999) was replacement 
of Silber by James Peyser (see Peyser, 1996, and Peyser, 1998), head of the 
reactionary Pioneer Institute, tied to school voucher and privatization movements, 
and retention of the compliant acting Commissioner David Driscoll. Since the 
Silber era, MCAS development has been closely monitored by Board of 
Education member Abigail Themstrom, a fellow of the Manhattan Institute, (Note 

20) and hired consultant Sandra Stotsky (see Stotsky, 1999), a writer for the 
Fordham Foundation and now an Associate Commissioner of Education. Both of 
these right-wing foundations have supported forms of school privatization. Four 
rounds of MCAS tests have now been administered, in the spring of 1998, 1999, 
2000 and 2001. The Board of Education has made the questions used in scoring 
available to the public, although they have not disclosed their standards for 
evaluating essay questions or all the details of their approach to computing scores. 
(Note 21) Students in religious-run and other private schools and students being 
taught at home are not required to take or pass MCAS tests. Bills have been filed 
but have not been enacted to include private schools in testing and to exclude 
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charter schools. A system of "school accountability" has been defined by the 
Department of Education (Mass. DoE, 1999b). It is based entirely on MCAS 
scores, a violation of Education Reform Act requirements. (Note 22) 

MCAS has been heavily promoted by a business-oriented group organized as 
Mass Insight Education and Research Institute, Inc., in Boston, founded in 1997 
by registered Massachusetts lobbyist William H. Guenther, who is its president. 
(Note 23) Guenther is also involved with three other public relations 
organizations. Mass Insight Corp., in Cambridge, Opinion Dynamics Corp., in 
Cambridge, and New England Economic Project, in Walpole. Mass Insight 
Education and Research Institute is a non-profit corporation that coordinates 
several policy groups and has close relationships with business and education 
executives. Leaders of its "Campaign for Higher Standards" have included Gloria 
Larson, former Mass. Secretary of Economic Affairs, the late John C. Rennie, 
former Chairman of the Massachusetts Business Alliance for Education and Vice- 
Chairman of AverStar, Inc., and Cathy Minehan, President of the Federal Reserve 
Bank of Boston. Leaders of its "Coalition for Higher Standards" include James 
Caradonio, Superintendent of Worcester Public Schools, and Thomas Payzant, 
Superintendent of Boston Public Schools. Its board of directors has included 
Maura Banta, Manager for External Programs at IBM Corporation, John Rennie, 
Abigail Themstrom, Senior Fellow at the Manhattan Institute and member of the 
Massachusetts Board of Education, and Bruce Tobey, Mayor of Gloucester. 
Financial supporters of Mass Insight Education and Research Institute include 
BankBoston (now FleetBoston Financial), State Street Corp., Bell Atlantic (now 
Verizon), Boston Edison (now an NSTAR division), Liberty Mutual Group, 
PricewaterhouseCoopers, Goodwin, Procter & Hoar, AverStar, Inc. (now a 
division of Titan Corp.), Gorton's Seafoods, Hewlett-Packard, IBM and Intel. 

Mass Insight publications promoting MCAS have been distributed to public 
schools through the Massachusetts Board of Education, (Note 24) and Mass 
Insight has received public funds for its services. Mass Insight has been cited in 
minutes of the Board of Education as a source of policy initiatives, (Note 25) 
including a proposal to use a score of 220 on tenth-grade language arts and 
mathematics tests as the initial "competency determination" for high-school 
graduation, which was adopted by the Board in November, 1999. Mass Insight 
presents a simple but misleading picture of MCAS, saying that it measures "skills 

that students will need after graduation?at college or on the job" (Why, 1999). 

No such significance has ever been demonstrated for MCAS or other state 
accountability tests. 

Massachusetts schools are often castigated by newspapers and politicians as 
mediocre, (Note 26) but actually they are superior. In the October, 1999, Boston 
Magazine, Jon Marcus wrote: 

"According to assessment tests and other measures, Massachusetts 
schools are among the nation's best. Students here rank fourth 
nationally in reading, sixth in math, and eighth in science on the 
National Assessment of Educational Progress, administered by the 
US Department of Education. They scored higher this spring in 
reading than 69 percent of their peers across the country on the Iowa 
Test of Basic Skills; a third of the state's third graders were at the 
advanced level, compared to 19 percent nationwide. A Boston 
College correlation of NAEP results with international tests found 
that in eighth-grade science Massachusetts students performed as 
well as, or better than, their counterparts in 40 out of 41 other 
countries, including Germany and Japan; only kids in Singapore were 
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rated higher. More students study algebra and upper-level math and 
science than the national average, and Massachusetts also has the 
fifth-lowest high school dropout rate, the nation's highest percentage 
of graduates who enroll in college, and the third-highest proportion 
of students who take the SAT. Massachusetts students' SAT results 
have risen steadily since 1994; last year, they were the highest in a 
decade." (From Marcus, 1999.) 

Such a contrast between political bombast and educational reality has become 
common. Part of the long record of declining SAT scores in the 1960s and 1970s, 
for example, had a straightforward cause, the rapidly expanding number of 
students taking the tests, including many low-income students and students with 
lower grades who would not have taken them in prior years (Koretz, 1992, 
Berliner and Biddle, 1995, Berliner and Biddle, 1996). When education 
researchers looked at comparable groups of students, SAT scores were gradually 
rising during much of this period; unadjusted averages began to rise as the growth 
in the number of test takers slowed. With the gratuitous abuse regularly heaped 
on public schools during this time, few members of the public would have 
guessed that some of the real trends in scores were positive. Even now, many 
politicians and most news media find the actual results inconvenient; they prefer 
simple, strident bashing of public schools, uncomplicated by facts. 

Many observers and columnists have commented on the complex language, 
mental tricks and obscure bits of knowledge found in MCAS questions (see, for 
example, Vaishnav, 2000, and Kohn, 2000). How elitist are the MCAS tests? One 
way to look at this is to ask the fraction of questions that must be answered to 
pass them and the fraction of students who cannot do this. Table Al-1 compares 
the current graduation level tests in Massachusetts (Note 27) (10th grade MCAS) 
with those in New York (Note 28) (the revised Regents series) and Texas (Note 
29) (the TAAS series). 



Table Al-1 

Comparison of State Achievement Tests 



State 


Typical percent of 
questions to pass 


Typical percent of 
students failing (Note 30) 


Texas 


70 


20 


New York 


55 


20 


Massachusetts 


40 


50 


Source ot data: see tex1 


Appendix 1 



Massachusetts, with by far the lowest passing score, has by far the highest rate of 
failure. Yet year after year, nationwide measures of academic performance rate 
Massachusetts students well ahead of those in New York and Texas. (Note 31) 
Passing an MCAS test says little about the education imparted through many 
years of schooling. On an MCAS tenth-grade mathematics test, the difference 
between passing and failing can be getting 24 questions right rather than 23 (see 
Mass. DoE, 1999d, and Mass. DoE, 1999c). 

A recent study performed by Catherine Horn and others at Boston College (Horn, 
et al., 2000) showed that a barely passing score on the tenth-grade MCAS math 
test was approximately equivalent to the 50th percentile score for the PSAT math 
test. Students taking the PSAT are aiming for college. Many are taking the test as 
part of applying for National Merit and other scholarships; they tend to be good 
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students. Therefore it should not be surprising when half or more of the general 
student population may "fail" the current tenth-grade MCAS math test. 

A leirge share of MCAS test questions is aimed at students with exceptional skills 
and Imowledge rather than at typical students. If Massachusetts designed tests to 
measure competence rather than mastery, it would be setting much higher passing 
percentages. If Massachusetts genuinely cared about assessing student skills and 
knowledge, it would satisfy Education Reform Act requirements (Note 32) calling 
for a "variety of assessment instruments," including "consideration of work 
samples, projects and portfolios," facilitating "authentic and direct gauges of 
student performance," and it would provide for circumstances of special 
education students, students entering the public schools from households that 
speak a first language other than standard English, (Note 33) and students whose 
immediate aims focus on employment rather than higher education. 

The Massachusetts Board of Education has ample access to information of this 
sort and has received many recommendations to improve its practices and make 
its system of assessments more realistic and fair. It has had more than $25 million 
to spend on developing MCAS (Szechenyi, 1998). It is also well aware that 
"high-stakes" testing systems in other states have sharply narrowed the school 
curriculum (Note 34) and increased the population of school dropouts, (Note 35) 
who are likely to be eligible only for the "MeJobs" of the future. So far, however, 
members of the Massachusetts Board of Education remain rigid, programmatic 
and hostile to facts that do not support their policies. Their attitude does not 
originate fi'om lack of information or resources. 

MCAS, like the other "achievement tests" used in state accountability systems, 
has never been shown to predict success in adult life to any greater degree than 
could be done with a knowledge of student backgrounds. Instead of trying to 
show practical significance the Board assumes it, in proposing to use this test as 
the sole state criterion to deny high-school diplomas and state college eligibility 
to low-scoring students, making it difficult for them to find responsible jobs and 
other forms of advancement. Students from households that already have the least 
suffer the most from such a system, tending to widen an economic gap between 
haves and have-nots in our society, already among the greatest of the industrial 
nations. 

Appendix 2: Massachusetts Vocational Schools 

Massachusetts municipal school districts support either jointly or individually 
more than 30 "vocational," "vocational-technical," "technical," "agricultural- 
technical" and "agricultural" high schools, (Note 36) most of which the 
Department of Education recognizes as separate school districts. Table A2-1 
includes the 29 vocational schools that are now operated as separate school 
districts plus the "technical" high school operated by the City of Boston. (Note 
37) 

Table A2-1 

1999 MCAS Grade 10 Math Test Scores for Vocational Schools 
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Vocational School 


1999 MCAS math 10 




Average 


Number 


Assabet Valley Voc. High 


211 


186 


Bay Path Reg. Voc. Tech. 


210 


245 


Blackstone Valley School 


213 


209 


Blue Hills Reg. Voc. Tech. 


213 


215 


Bristol County Agr. High 


212 


100 


Bristol Plymouth Voc. Tech. 


208 


207 


Cape Cod Reg. Voc. Tech. High 


214 


171 


Charles McCann Voc. Tech. 


217 


105 


Diman Reg. Voc. Tech. High 


208 


312 


Essex Agr. & Tech. Inst. 


212 


80 


Franklin County Tech. 


214 


115 


Gr. Lowell Reg. Voc. Tech. 


205 


466 


Gr. New Bedford Voc. Tech. 


208 


497 


Greater Lawrence Tech. 


205 


320 


Joseph Keefe Tech. High 


208 


160 


Madison Park Tech. High 


202 


289 


Minute Man Voc. Tech. High 


216 


199 


Montachusett Voc. Tech. 


210 


257 


Nashoba Valley Tech. High 


208 


121 


Norfolk County Agr. 


214 


115 


North Shore Tech. High 


210 


118 


Northeast Metro. Reg. Voc. 


206 


297 


Old Colony Reg. Voc. Tech. 


210 


127 


Pathfinder Voc. Tech. 


210 


134 


Shawsheen Valley Voc. Tech. 


209 


281 


So. Shore Voc. Tech. High 


212 


130 


Southeastern Reg. Voc. Tech. 


209 


302 


Tri County Reg. Voc. Tech. 


212 


215 


Upper Cape Cod Tech. 


206 


134 


Whittier Reg. Voc. 


204 


352 


Averages 


210 


215 



Source ot data: Mass. UoJi, 2UUUti 



Vocational schools typically provide instruction for grades 9 through 12 and 
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devote about half their instructional time to traditional academics and about half 
to vocational training. Some communities, including Cambridge, Quincy, Revere 
and Waltham, provide vocational education in the same facilities as academic 
programs. In contrast to academic high school programs, which mainly concern 
themselves with preparing students for college, vocational programs train 
students for specific occupations, and their faculty tend to rate themselves 
according to graduates' success in finding satisfactory employment in those 
occupations. 

Vocational schools, represented through the Massachusetts Association of 
Vocational School Administrators, have presented the Massachusetts legislature 
with a bill to decouple those schools from MCAS and substitute a special 
examination system based on the occupational categories for which they provide 
training, sponsored by Sen. David Magnani of Framingham (Magnani, 2000). 
They are well aware, as the foregoing table shows, that their students score far 
below state averages on MCAS tests; but they claim that MCAS tests are directed 
toward a curriculum that they do not teach and cannot teach without weakening 
their key programs. Table A2-2 identifies some of the major sources of 
information for Massachusetts vocational schools available on the Internet. 

Table A2-2 

Massachusetts Vocational School Information Sources 
Massachusetts Association of Vocational School Administrators 
Bay Path Regional Vocational Technical High School, Charlton, MA 
Blackstone Valley Regional Vocational Technical High School, Upton, MA 
Bristol-Plymouth Regional Technical School District, Taunton, MA 
Bluehills Regional Technical High School, Canton, MA 
Diman Regional Vocational Technical High School, Fall River, MA 
Greater Lowell Technical High School, Tyngsboro, MA 

Greater New Bedford Regional Vocational Technical High School, New Bedford, 
MA 

Lower Pioneer Valley Educational Collaborative, East Longmeadow, MA 
Minuteman Science-Technology High School, Lexington, MA 
Northeast Metropolitan Regional Vocational School, Wakefield, MA 
Old Colony Regional Vocational Technical High School, Rochester, MA 
Shawsheen Valley Technical High School. Billerica, MA 
Tri-County Regional Vocational Technical School District, Franklin, MA 
William J. Dean Technical High School, Holyoke, MA 
Whittier Regional Vocational Technical High School, Haverhill, MA 
Worcester Vocational High School, Worcester, MA 
Population trends at Massachusetts Regional Vocational School Districts 
Source of data: Massachusetts Association of Vocational School Administrators 

Appendix 3: Metropolitan Boston MCAS Mathematics Scores 



Table A3-1 

1999 MCAS Grade 10 Math Test Scores for Boston-area Schools 





2000 


1999 


1998 


City or 


High School 


Adj 


Adj 


Adj 


Adj 


Adj 


Adj 


Town 




Avg 


Num 


Avg 


Num 


Avg 


Num 


Arlington 


Arlington 


239 


231 


234 


246 


230 


251 


Belmont 


Belmont 


247 


254 


243 


229 


245 


223 


Boston 


Boston High 


210 


111 


204 


132 


203 


168 


Boston 


Brighton 


208 


233 


205 


216 


203 


186 
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Boston 


Charlestown 


210 


175 


206 


OO 


207 


156 


Boston 


Dorchester 


205 


134 


204 


121 


206 


128 


Boston 


East Boston 


210 


270 


205 


206 


204 


225 


Boston 


Hyde Park 


204 


129 


203 


99 


203 


249 


Boston 


Jeremiah 

Burke 


211 


112 


208 


145 


204 


85 


Boston 


South Boston 


210 


177 


205 


159 


205 


182 


Boston 


The English 
High 


208 


200 


204 


203 


203 


230 


Boston 


West 

Roxbury 


210 


211 


205 


269 


202 


149 


Boston 

Exam 


Boston Latin 


262 


382 


254 


399 


246 


417 


Boston 

Exam 


Latin 

Academy 


252 


248 


233 


245 


230 


222 


Boston 

Exam 


O'Bryant 

Science 


235 


222 


227 


258 


223 


248 


Braintree 


Braintree 


241 


319 


228 


306 


229 


324 


Brookline 


Brookline 


245 


429 


240 


416 


238 


406 


Cambridge 


Rindge & 
Latin* 


219 


331 


220 


423 


222 


404 


Chelsea 


Chelsea 


219 


203 


216 


207 


215 


242 


Dedham 


Dedham 


238 


180 


227 


174 


226 


172 


Everett 


Everett* 


218 


325 


221 


295 


216 


277 


Lexington 


Lexington 


243 


407 


238 


368 


239 


362 


Lynn 


Classical 


223 


294 


216 


281 


216 


246 


Lynn 


English 


224 


285 


213 


247 


212 


257 


Malden 


Malden 


225 


305 


221 


286 


219 


265 


Marblehead 


Marblehead 


241 


169 


232 


145 


237 


187 


Medford 


Medford* 


228 


228 


221 


231 


219 


232 


Melrose 


Melrose 


228 


214 


226 


245 


225 


243 


Milton 


Milton 


232 


241 


228 


208 


229 


204 


Newton 


North* 


248 


521 


239 


468 


242 


468 


Newton 


South 


252 


309 


242 


285 


240 


282 


Peabody 


Veterans* 


226 


426 


220 


354 


219 


372 


Quincy 


North Quincy 


234 


301 


227 


303 


224 


294 


Quincy 


Quincy* 


220 


238 


212 


200 


214 


279 


Revere 


Revere* 


224 


251 


218 


306 


216 


286 


Salem 


Salem* 


224 


234 


220 


243 


219 


257 


Saugus 


Saugus 


233 


193 


226 


226 


222 


232 


Somerville 


Somerville* 


218 


328 


216 


338 


217 


307 


Stoneham 


Stoneham 


235 


171 


227 


169 


233 


171 


Swampscott 


Swampscott 


239 


195 


240 


161 


224 


181 


Wakefield 


Memorial 


231 


192 


227 


228 


230 


243 


Waltham 


Waltham* 


228 


344 


220 


339 


221 


384 
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Watertown 


Watertown 


233 


164 


231 


159 


221 


156 


Weymouth 


Weymouth* 


227 


466 


222 


455 


222 


433 


Winchester 


Winchester 


249 


225 


243 


198 


244 


178 


Winthrop 


Winthrop* 


227 


120 


223 


118 


218 


162 


Woburn 


Woburn 


237 


276 


226 


285 


226 


240 


Source ot d 


lata: see text, / 


Append 


lix 3 



Schools marked with asterisks (*) in Table A3-1 are those providing vocational 
education in the same facility as academic programs. 

Data in Table A3-1 are tenth-grade MCAS mathematics test scores for 1998- 
2000, averaged by schools, and numbers of test participants per school, obtained 
from the Massachusetts Department of Education (Mass. DoE, 2000h) and 
adjusted for percentages of students enrolled in schools but not taking the test. 
Adjustment formulas are as follows: 

Nadl-NC-P/lOO) 

S,dj = a00S-200P,)/(100-P^), 

where 

N is the number of enrolled students 



S is the average score on test, per 
Department of Education 

Pg is the percentage of "absent" students 
(not taking test) 



NadJ is the adjusted number of students 
(number taking test) 



Sgdj the adjusted average score (only 
students taking test) 



This procedure cannot adjust correctly for students absent for some but not all test 
sections. The Department has not published such information for the years 1998- 
2000. Adjusted results are rounded to the nearest integer. 

Appendix 4: Metropolitan Boston School Characteristics 

Table A4-1 

Data for Boston-area School Characteristics 
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City or Town 


High School 


Pop./ 

Grade 


% 

African 

Amer. 


% 

Asian 

or 

Pac. 

Isl. 


% 

Hispanic/ 

Latino 


% 

Lim. 

Eng. 

Pr. 


% 

Free 

Lunch 


9, lo- 
ll, 12 
Reduct. 


Per 

Cap. 

Income 


Arlington 


Arlington 


259 


6.7 


4.3 


2.7 


0.7 


8.1 


0.2 


21.4 


Belmont 


Belmont 


241 


2.8 


5.6 


0.2 


0.4 


2.5 


10.4 


26.8 


Boston 


Boston 

High 


183 


56.5 


3.3 


26.5 


14.4 


64.3 


4.0 


15.6 


Boston 


Brighton 


267 


49.5 


7.9 


35.1 


32.0 


62.1 


29.4 


15.6 


Boston 


Charlestown 


288 


36.1 


22.5 


27.3 


36.9 


59.9 


14.3 


15.6 


Boston 


Dorchester 


249 


66.7 


2.7 


25.7 


29.7 


51.4 


26.2 


15.6 


Boston 


East Boston 


317 


24.0 


6.7 


40.3 


29.8 


62.9 


47.0 


15.6 


Boston 


Hyde Park 


220 


72.0 


1.6 


18.4 


23.5 


53.6 


15.7 


15.6 


Boston 


Jeremiah 

Burke 


176 


86.0 


3.6 


26.1 


32.9 


56.7 


20.0 


15.6 


Boston 


South 

Boston 


267 


46.7 


18.4 


16.6 


26.1 


44.6 


24.3 


15.6 


Boston 


The English 
High 


332 


50.1 


1.9 


36.9 


45.5 


62.0 


24.2 


15.6 


Boston 


West 

Roxbury 


328 


64.7 


2.4 


21.1 


28.7 


64.3 


18.8 


15.6 


Boston 

Exam 


Boston 

Latin 


398 


18.5 


22.7 


8.4 


0.6 


29.3 


16.2 


15.6 


Boston 

Exam 


Latin 

Academy 


255 


27.7 


20.2 


7.7 


2.8 


40.0 


28.3 


15.6 


Boston 

Exam 


O'Bryant 

Science 


252 


45.1 


31.8 


13.4 


18.3 


48.7 


23.0 


15.6 


Braintree 


Braintree 


337 


3.6 


3.0 


1.0 


0.4 


4.2 


12.0 


18.6 


Brookline 


Brookline 


428 


12.5 


13.7 


3.9 


2.6 


15.2 


8.7 


29.0 


Cambridge 


Rindge & 
Latin* 


478 


37.4 


7.6 


14.2 


7.7 


14.8 


7.7 


19.9 


Chelsea 


Chelsea 


281 


8.5 


10.9 


63.7 


15.1 


57.8 


39.1 


11.6 


Dedham 


Dedham 


179 


1.6 


2.2 


4.1 


1.7 


3.2 


22.6 


19.0 


Everett 


Everett* 


336 


8.4 


3.6 


11.2 


5.2 


23.0 


21.1 


14.2 


Lexington 


Lexington 


386 


7.3 


15.0 


0.6 


1.1 


3.9 


12.3 


30.7 


Lynn 


Classical 


282 


17.5 


24.0 


14.2 


14.6 


38.8 


26.5 


13.0 


Lynn 


English 


327 


11.3 


14.2 


23.0 


20.1 


23.2 


20.5 


13.0 


Malden 


Malden 


346 


13.0 


21.0 


6.8 


10.5 


17.6 


15.6 


15.8 


Marblehead 


Marblehead 


178 


3.9 


0.7 


0.6 


1.1 


3.5 


-3.7 


30.6 


Medford 


Medford* 


266 


11.2 


3.9 


1.3 


2.9 


6.0 


-1.7 


16.9 


Melrose 


Melrose 


242 


4.2 


1.3 


1.2 


0.7 


5.4 


-2.9 


20.2 


Milton 


Milton 


232 


15.7 


2.2 


1.3 


0.8 


4.2 


7.9 


22.4 


Newton 


North* 


495 


5.9 


7.8 


1.7 


2.4 


3.0 


8.4 


28.8 


Newton 


South 


300 


3.8 


11.1 


1.0 


1.9 


1.6 


8.2 


28.8 


Peabody 


Veterans* 


418 


0.8 


1.2 


7.5 


2.0 


8.4 


20.0 


17.0 


Quincy 


North 


325 


0.6 


26.5 


1.2 


0.0 


14.4 


19.5 


17.4 
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Quincy 



Quincy 


Quincy* 


282 


4.7 


14.1 


3.6 


9.0 


15.7 


4.7 


17.4 


Revere 


Revere* 


339 


3.6 


15.4 


7.9 


4.6 


21.6 


26.4 


14.7 


Salem 


Salem* 


279 


5.1 


2.2 


20.0 


6.8 


26.3 


31.4 


16.2 


Saugus 


Saugus 


221 


0.7 


1.9 


0.7 


0.0 


7.9 


17.2 


17.8 


Somerville 


Somerville* 


467 


17.1 


5.9 


14.3 


15.4 


52.8 


24.6 


15.2 


Stoneham 


Stoneham 


191 


0.5 


1.4 


1.2 


0.0 


7.2 


3.3 


18.2 


Swampscott 


Swampscott 


184 


1.5 


0.3 


1.6 


1.4 


2.8 


4.0 


25.6 


Wakefield 


Memorial 


240 


1.9 


1.5 


0.4 


0.1 


5.8 


5.7 


19.0 


Waltham 


Waltham* 


381 


9.0 


7.1 


15.1 


3.4 


15.9 


-2.5 


16.8 


Watertown 


Watertown 


184 


3.7 


3.4 


4.2 


3.1 


14.2 


6.1 


20.4 


Weymouth 


Weymouth* 


464 


2.1 


1.8 


1.0 


0.1 


6.3 


16.8 


18.4 


Winchester 


Winchester 


209 


1.6 


4.0 


0.5 


0.0 


2.3 


16.9 


30.6 


Winthrop 


Winthrop* 


140 


2.3 


2.1 


2.7 


1.6 


4.1 


-12.9 


17.9 


Woburn 


Wobum 


284 


2.0 


2.1 


3.3 


1.6 


7.7 


18.7 


18.2 



Source ot data: see text, Appendix 4 



Data in Table A4-1, except per-capita income, are gathered from the reports of 
public schools to the Massachusetts Department of Education for the school year 
ended June 30, 1999, and published by the Department in its school profiles 
(Mass. DoE, 2000f). Data for per-capita community income in $ 1,000s are from 
the Massachusetts Department of Revenue (Mass. DoR, 1999), based on the 1990 
US Census of Population and Housing. Schools marked with asterisks (*) are 
those providing vocational education in the same facility as academic programs. 
All numbers except average student population per grade and per-capita income 
are percents. Grade size reduction has been calculated as percent decrease in the 
grades 1 1+12 school population as compared with grades 9+10. School district 
reports (Mass DoE, 1999a) include the following: (Note 38) 

Foundation Enrollment and Student Attendance Reports, reporting foundation 
enrollment and student attendance as of October 1, submitted to the Department 
by each district no later than December 15 during each school year. 

Individual School Reports and School System Summary Reports, reporting on 
student enrollments and classifications as of October 1, including gender, "racial" 
or "ethnic," limited English proficiency and low-income status, submitted to the 
Department by each district no later than January 3 1 during each school year. 

School Attending Children Report, counting all school-age residents of a district 
classified by grade and type of school attended, as of January 1 , submitted to the 
Department by each district no later than April 30 during each school year. 

Year-End School Indicator Reports, submitted to the Department by each district 
no later than the August 1 5 following each school year. 

End-of-Year Pupil and Financial Reports (EOYR), submitted to the Department 
by each district no later than the September 30 following each school year. An 
EOYR currently consists of the following schedules: 

1 . Revenue and Expenditure Summary 

2. Assessments Received From Member Towns or Cities or 
Regional School Districts 

3. Instructional Services By Grade Level 
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4. Special Education Expenditures by Prototype 

7. Pupil Transportation Reimbursement 

8. Professional Development 

1 1 . Pupil Membership Summary 

13. Staff Data By Major Program Area Instructional Programs 
16. Pupils-Attendance Data 
19. Armual School Budget 

As of July 1, 2000, the Department requires school districts to compile data on 
instructional costs at the school building level. New financial reporting will use a 
revised and unifonn chart of accounts in the EOYR due September 30, 2002. 
However, as previously noted, current school profiles do not include expenditures 
for schools. The Department has a long cycle for publication of data. Final 
summary data on per-pupil expenditures for the school year ending June 30, 1999, 
were released January 20, 2001. 
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Notes 

*The terms "racial" and "ethnic" will be used here in the senses of state and 
federal regulations, although they describe populations not always closely related 
by genetic or cultural backgrounds. 

^For the fall of 1997, 85 percent of Massachusetts elementary and secondary 
school students were in public schools and 1 5 percent in private schools of all 
types. See National Center, 2001, Tables 39 and 64. 

^On the motion of Chairman John Silber, Mass. DoE, 1998a. 

‘^Massachusetts General Laws, Chapter 69 (Powers), Sections IB and ID. The 
Department of Education publishes "curriculum frameworks" with only the legal 
status of guidelines and recommendations. See Mass. DoE, 2000J. 

^Massachusetts General Laws, Chapter 69 (Powers), Section ID, paragraph (i). 

®See Mass. DoE, 2000i, Chapter 12, for procedures to equate scores across test 
series. This document does not describe any procedures for relating test scores to 
education content as delivered in Massachusetts public schools. 

^Model and MANOVA evaluations performed with Statistica, Version 5.5, 
StatSoft, Tulsa, OK. 

^Compare "promoting power" or "holding power" in Balfanz and Legters, 2001. 
For issues concerning availability of accurate data on public school dropout rates, 
see Kaufman, 2001 . 



httn*//en?ifi pHii/pnaa/v0n49/ 



417 



10/30/01 




EPAA Vol. 9 No. 4...: Significance of Test-based Ratings for Metropolitan Boston School Page 47 of 60 



^Other school characteristics also deserve consideration. Boston may have 
enrollment patterns for high-school mathematics classes differing from suburban 
norms. Ninth-grade students classified as "Asian" have been found more than 
twice as likely as average students to be enrolled in traditional "high school 
mathematics" courses, while students classified as "Hispanic" were found less 
likely than average to be taking such courses. (Anne Wheelock, personal 
communication, January, 2001). 

'®It is possible that "limited English proficiency" acts as a proxy for other, 
potentially stronger factors, much as certain "racial" or "ethnic" percentages 
appear to act as proxies for income factors in these data. 

* ^ Other studies have reported significant "racial" or "ethnic" test score differences 
for individuals after adjusting for incomes. For one survey and an evaluation, see 
Hedges and Nowell, 1998, pp. 149-181. However, few studies have considered 
school ratings based on scholastic achievement tests given in the mid-teen years 
and have used commensurate data for a geographical cluster of communities that 
were each fairly homogeneous. 

*^See Mass. DoE, 1999b. For the first published ratings, see Mass. DoE, 2001. 

For potential penalties, see Mass. DoE, 2000b. 

*^See "Relationships between student MCAS performance and courses taken," in 
Brian Gong, Relationships Between Student Performance on the MCAS and Other 
Tests— Collaborating District A, National Center for the Improvement of 
Educational Assessment, Inc., March, 1999, pp. 39-48, available at 
http://www.nciea.org/uploadCFA /MA%20report.pdf. 

''^See Cremin, 1970, Kaestle, 1983, Tyack, 1974, and Tyack and Cuban, 1995. 

'^Referred to herein as the "Business Alliance." Not to be confused with the 
Massachusetts Global Business Alliance, Business Education Alliance, Regional 
Education and Business Alliance, or any of several other organizations with 
similar names. 

*^See Mass. Business, 1991. Principal author of the sections dealing with school 
finance was economist Edward Moskovitch, a former Massachusetts chief budget 
director and executive director of the Massachusetts Municipal Association. The 
title of this document should not be confused with the Every Child a Winner 
(ECAW) school programs and educational games produced by Martha F. Owens 
and Susan B. Rockett, beginning in 1974, and now distributed by Educational 
Excellence, Inc., of Ocilla, GA; there appears to be no connection; see 
http://www.ed.gov/pubs/EPTW/ep tw9/eptw9b.html. 

'^"Students in the graduating class of 2003 shall meet or exceed the.. .score of 220 
on both the English Language Arts and the Mathematics MCAS grade 10 tests...." 
See Mass. DoE, 1999f 

*^See Mass. DoE, 1999c, Chapter 8, Scoring, and Chapter 10, Scaling. 

*^Mass. DoE, 2000a. Also see Mass DoE, 1998c, and Mass. DoE, 1999c. 
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^^See Antonucci, 1995, for the following: "As students are better matched to their 
institutions," [says Abigail Thernstrom], "as they cascade to places where they are 
prepared to the average level, the graduation rates should go up for minorities." 

^*For most of the available information, see Mass DoE, 1999c. 

^^Massachusetts General Laws, Chapter 69 (Powers), Section II (second 
paragraph). Also see Allen, 1999. 

^^Information on Mass Insight is available at 
http://www.massinsight.coin/meri/index. html. 

^^For example, a "Starting Now" brochure for parents. See Driscoll, 1998a, and 
Driscoll, 1999b. The versions distributed include: Fall 1998, Spring 1999, 
Manufacturing and Processing Industry Spotlight, Fall 1999, Spring 2000, Fall 
2000 and Spring 2001 -all available from the Massachusetts Department of 
Education at http://www.doe.mass.edu/mcas/p ubs/other_pub.html. 

^^See the following examples: "Recent survey by Mass Insight," cited in Goals 
2000 Five Year Master Plan, Goal 5, "Create a statewide infrastructure," March, 
1995; "Mass Insight report on education reform," minutes of March 22, 1996; 
presentations on the graduation requirement by Bill Guenther, Mass Insight 
Education and by Susan Kiernan and John Lozada, Mass Insight's Campaign for 
Higher Standards, minutes of November 23, 1999. 

^^"A majority of parents surveyed said they would rather pay out of their pockets 
to send their children to private or parochial schools than send them to Haverhill 
schools" (quoted in Grodsky, 1999). "Almost every school system is loaded with 
incompetent administrators" (John Silber, quoted in Drew and Suhler, 1998). "We 
should change the teacher tenure law so we can dismiss incompetent 
teachers" (Lamar Alexander, former Governor of Tennessee and US Secretary of 
Education, quoted in Patch and Wallace-Wells, 1998). "The dismissal of 
incompetent teachers is made almost impossible in some communities by such 
over-zealous delirium on the part of good people" (from Samuel P. Orth, the 
author of A History of Cleveland, in Orth, 1909). 

^^For passing scores, see Mass. DoE, 1999f, and Mass. DoE, 1999c, Chapter 8, 
Scoring, and Chapter 10, Scaling. For typical scores from 1998 and 1999, 
statewide, see. Mass. 1999e, Executive Summary. 

^^For passing scores, see NY DoE, 2000a. For typical scores from 1999, see NY 
DoE, 2000b. 

^^For passing scores, see TX DoE, 2000a. For typical scores from 2000, see TX 
DoE, 2000b. 

^®In some formats. New York and Texas report lower failure percentages because 
they exclude some students or they include students who pass after multiple 
attempts. 
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^*See, for example, National Assessment, 1998, and similar summary tables from 
other years. 

^^Massachusetts General Laws, Chapter 69 (Powers), Section II (second 
paragraph). 

^^See Madaus and Clarke, 1998. 

^'^See McNeil, 2000, and Heubert and Hauser, 1999. 

^^See Clarke, et al., 2000, and Haney, 2000. 

^^As required by Massachusetts General Laws, Chapter 74 (Vocational 
Education), Section 7, and Mass. DoE, 1997. 

^^Data from Mass. DoE, 2000h. 

^®Also see EdTech, Massachusetts Department of Education, at 
http://www.doe.mass.edu/edtech. 
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Abstract 

This study presents a time-series evidence on the timing 
and degree of feedback relationship between participation 
in education and income growth in Hawaii. Using the 
unrestricted vector autoregression approach and two 
related measures of linear dependence and feedback, the 
results suggest that across all educational levels, i.e., K-12 
and tertiary, participation in public education could Ije a 
good predictor of income growth in Hawaii. However, 
decomposing the feedback effect by frequency suggests 
that the dominance of public education over private 
education in explaining the variation in income growth to 
be concentrated mainly on the short-run to medium-run 
for tertiary level and long-run to permanent effect for K- 
12 level. Hawaii state legislature and educators should 
perhaps take these results as a motivation not to ignore the 
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problems plaguing Hawaii's public schools but should 
work towards greater improvement and support for public 
education given its predicted significant overall 
contribution to the Hawaiian economy. 



Introduction 

The lackluster condition of the Hawaiian economy when 
compared with the economic expansion in the mainland state 
economies since late 1991 led the Hawaiian legislature to reassess the 
economy's traditional sole dependence on the tourism industry. To 
help revive the economy, the state government focused on educational 
reform as one of their priorities. Hawaii needs to build its human 
capital stock to be an active player in the new information or 
knowledge-based global economy. To help ensure the availability of 
educated and skilled human resources, the presence of dynamic 
research and teaching institutions is eminent. However, despite the 
pronounced good intentions and plans made by the state government, a 
growing number of Hawaii residents realize that not enough is being 
done. Based on a statewide survey, residents are generally 
disappointed about the economy and the condition of education. In 
fact, with the dreary statewide economic performance comes difficult 
choices and the need for re-allocation of resources. So, where does 
public and private education stand in all this? 

In this study, an empirical investigation is done to assess and 
compare the relative contribution of public and private schools to 
Hawaii's economy. This paper presents a time-series evidence on the 
timing and degree of feedback relationship between participation in 
education and income growth in Hawaii. The empirical investigation 
uses two feedback methods to measure the degree of dependence or 
the extent of feedback between data series and a related measure to 
distinguish between short-run and long-run effects of a given 
innovation or shock. This study is intended to contribute to a better 
understanding of the condition and quality of the educational system in 
Hawaii. Also, the findings of this study may have important 
implications for directing resources or investment in education and 
shaping of Hawaii's educational policy in the future. 

Education in Hawaii: An Overview 

The establishment of early schools in Hawaii was due to the 
efforts of missionaries in the 1 840s. Public education was first 
instituted on October 15, 1840 with mandatory attendance of children 
from ages four to fourteen. The upkeep of earlier statistics on 
education in Hawaii was difficult because of numerous changes on its 
classifications. For instance, the compulsory age for school attendance 
went through six changes: ages four to fourteen in 1840, six to sixteen 
in 1859, six to fifteen in 1865, six to fourteen in 1923, six to sixteen in 
1937 and finally six to eighteen in 1965. Second^ education during 
the early monarchy years in Hawaii was also limited and left largely to 
government-subsidized private schools while, higher education was 
developed only in the twentieth century. 

Hawaii became the 50th state on Aug 21, 1959. In 1960, 46% of 
the population had four years or more of high school training while 
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only 9% had four years or more of college training. As of 1998, 84% 
of the population are high school graduates while 24% have bachelor's 
or advanced degree. 

Overview of School Enrollment and Educational Resources 

As summarized in Table 1 , the participation in Hawaii's formal 
public education at the level of kindergarten to grade 12 had its biggest 
growth increase in the 1960s; while private schools had its biggest 
increase in enrollment in the following decade. Enrollment in K-12 
exhibited contrasting trend for public and private schools, i.e., when 
public institutions experienced positive growth, the private institutions 
suffered a negative growth and vice versa. 

Table 1 

Average Growth Rate in K-12 and Tertiary Enrollment, 
Number of Schools and Teachers 





1960-69 


1970-79 


1980-89 


1990-99 


Public K-12 Institutions 


Enrollment 


2.42 


-0.58 


0.095 


0.91 


Schools 


0.44 


0.59 


0.389 


0.669* 


Teachers 


4.49 ' 


1.05 


1.34 


2.051* 


Private K-12 Instit 


utions 


Enrollment 


1.12 


1.32 


-0.403 


0.18 


Schools 


2.63- 


1.38 


-0.25 


-1.064** 


Teachers 


3.36 


3.03 


2.62 


0.177** 


Tertiary Enrollment 


Public 


10.37 


1.31 


-1.13 


-0.455 


Private 


9.43 


9.65 


5.59 


2.60 



Note: Figures indicated with * refer only to 1990-97 while those with 
** refer only to 1990-96. 

For some years, the number of K-12 schools established does 
not seem to follow the enrollment trend. In particular, the number of 
public schools in the island posted an increase of 0.6% in the 1970s at 
a time when it experienced a comparable 0.6% decline in enrollment. 
Conversely, at a time of recovery in enrollment, the number of public 
schools established continued to decline. The number of private 
schools recorded big increases during the 1960s and 1970s but was 
drastically reversed in the 1980s and 1990s. In terms of school 
resources, both public and private schools had their biggest growth 
increase in hiring teachers during the 1960s. However, in terms of 
average number of pupil per teacher, private schools do a better job 
than public schools in providing small classes due in part to private 
schools continued bigger increases in hiring teachers. The public 
school system also continue to be plagued by other problems or 
concerns such as low test scores, aging facilities and low teacher 
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morale. 

For tertiary level, private universities exhibited continuous 
positive growth in enrollment from 1960 to 1999. In contrast, the 
public university suffered a drastic drop in enrollment in the 1970s 
relative to the previous decade, and turned into a negative growth in 
the 1980s and 1990s. This downward trend in enrollment may not 
seem surprising given that the state funding for the public university 
system dropped 19% in the past ten years. In fact, a national survey 
spotlighted Hawaii as the state with the largest loss in state support for 
higher education in 1998-99. Budget cuts have forced some programs 
to close or cease operation. A state law that sets a $352 million floor in 
state funding for the University of Hawaii (UH) was amended by the 
legislature wherein they are now to provide only for an appropriation 
ranging from 60-80% of funds required in addition to tuition. 
Beginning in 1995-1996, UH was allowed to keep tuition fees which 
formerly go into state general fund. Despite this change, the state 
university system still finds their resources constrained that they have 
to resort to increasing tuition fees which took a toll in their enrollment. 

Data and Description of Methodology 

Data on school enrollment and per capita Gross State Product 
were taken from The State of Hawaii Data Book, various issues, Dept, 
of Planning and Economic Development. Earliest available data for 
private universities were recorded in 1955 and were taken from 
various sources such as Historical Statistics of Hawaii by Robert 
Schmitt (1977) and Hawaii State Department of Education records. 
Private universities in Hawaii primarily consists of Bringham Young 
University of Honolulu, Chaminade University and Hawaii Pacific 
University. In this study, data on public university account only for 
enrollment at the University of Hawaii at Manoa which is the biggest 
institution in the state university system. Data on the number of K-12 
schools and teachers for both public and private institutions were taken 
from the Hawaii State Department of Education records. Given the 
availability of relevant data, this study covers the period of 1958 to 
1999. 

Given that a number of models are consistent with observed 
correlation between human capital and income growth, I used the 
unrestricted vector autoregression (VAR) approach to model the 
dynamic relationship among pertinent variables in order to minimize 
specification error. The VAR approach avoids the need for tight 
structural modeling by treating variables in a system as a function of 
all lagged values of all of the endogenous variables in the system 
(Hamilton, 1994). It uses only past regularities and historical patterns 
in the data as a basis for forecasting. In this study, a three-variable 
autoregressive system is used. The variables include income growth as 
proxied by the growth rate of real gross state product per capita, 
enrollment figures at different levels, i.e., K to 12 and higher education 
from both public and private schools to serve as proxies for human 
capital stock. A lag length of four years is used for all variables as 
suggested by the likelihood ratio test done. Also, based on the unit root 
tests conducted (Dickey, D. & Fuller, 1979; Kwiatkowski, D., Phillips, 
Schmidt & Shin, 1992; Phillips & Perron, 1988), the stationarity of 
some data series are inconclusive. Hence, the empirical investigation 
uses the data series in both levels and first differences or in percentage 
change. , 
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The details of the two related measures of linear dependence and 
feedback used in this study can be found in the Appendix. To measure 
the degree of dependence or the extent of various kinds of feedback 
between income growth and participation in education as measured by 
school enrollment, I used Geweke’s (1982) bi-variate feedback 
method. The feedl?ack measures are non-negative and zero only when 
feedback or causality of the relevant type is not present. A simple 
transformation of each feedback measure gives the reduction in the 
prediction error variance. Also, to distinguish between short-run and 
long-run effects of a given shock, I decomposed the feedback by 
frequency using McGarvey's (1985) methodology. I used this method 
on an expanded three- variable VAR system. 

Empirical Results and Data Analysis 

The bi-variate feedback results using Geweke's method are 
shown in Table 2. The results suggest that both in terms of levels and 
first differences, the magnitude of linear feedback from participation in 
K-12 private education to income growth to be about five times greater 
than the feedback from public enrollment to income growth. However, 
in terms of higher education, the magnitude of feedback from the 
public university is bigger than the feedback from participation in 
private universities. Also, at all educational levels (i.e., K-12 and 
tertiary), the feedback from public education to income growth 
remains bigger than the feedback from private education. This result 
may suggest that in Hawaii, participation in public education could be 
a good predictor of income growth. 

Table 2 

Feedback from Participation in Education to Income 

Growth 



Kto 12 


In levels 


In Percentage Change 


Public 


0.0852 (8.17%) 


0.0994 (9.46%) 


Private 


0.4997 (39.33%) 


0.5033 (39.55%) 


Higher 


Education 


Public 


0.2477 (21.94%) 


0.0743 (7.16%) 


Private 


0.0496 (4.84%) 


0.0459 (4.49%) 


All educational Levels 


Public 


0.1683(15.49%) 


0.0832 (7.98%) 


Private 


0.0788 (7.57%) 


0.0782 (7.52%) 



In order to have a better idea of an innovation's short-run versus 
long-run effects, the feedback measure is decomposed by frequency 
bands. Also, the bi-variate system is extended to a three-variable 
system and uses the ordering of 'growth prior to public education prior 
to private education' in the Choleski decomposition. Although the 
feedback measure is consistent, McGarvey showed that, in small 
samples, the feedback measure is biased upward. Hence, the Monte 
Carlo simulation method is used to derive bias-adjusted feedback 
estimates. Table 3 summarizes the adjusted estimates and figures 
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enclosed in parentheses pertain to the proportion of variance explained 
by a corresponding shock to a series. 



Table 3 

Feedback from Participation in 
Education to Growth by Frequency Levels 



In levels 


Private K-I2 


Public K-12 


Permanent 


0.0002 (0.02%) 


0.0015(0.15%) 


Long-run 


0.024 (2.36%) 


0.132 (12.40%) 


Medium-run 


0.174(15.97%) 


0.056 (5.49%) 


Short-run 


0.832 (56.48%) 


0.043 (4.24%) 


Overall 


0.271 (23.74%) 


0.061 (5.90%) 


In Percentage Change 


Private K-12 


Public K-12 


Permanent 


0.222(19.91%) 


1.124(67.51%) 


Long-run 


0.1281 (12.03%) 


0.376 (31.32%) 


Medium-run 


0.4108 (33.69%) 


0.0068 (0.68%) 


Short-run 


0.8790 (58.48%) 


0.0893 (8.54%) 


Overall 


0.516(40.29%) 


0.045 (4.40%) 


In Levels 


Private Universities 


Public University 


Permanent 


0.0966 (9.21%) 


0.0492 (4.79%) 


Long-run 


0.0547 (5.32%) 


0.2792 (24.36%) 


Medium-run 


0.0104(1.03%) 


0.1525 (14.15%) 


Short-run 


0.0114(1.13%) 


0.1087(10.30%) 


Overall 


0.0161 (1.59%) 


0.1526(14.15%) 


In Percentage Change 


Private Universities 


Public University 


Permanent 


0.0379 (3.72%) 


0.0028 (0.28%) 


Long-run 


0.0417 (4.08%) 


0.0123 (1.22%) 


Medium-run 


0.0612(5.94%) 


0.0797 (7.66%) 


Short-run 


0.0078 (0.78%) 


0.0709 (6.85%) 


Overall 


0.0405 (3.96%) 


0.0664 (6.43%) 



In Levels 


Private 


Public 


Permanent 


0.00012 (0.012%) 


0.1537(14.24%) 


Long-run 


0.0230 (2.28%) 


0.1461 (13.60%) 


Medium-run 


0.0048 (0.48%) 


0.1256(11.80%) 


Short-run 


0.0568 (5.52%) 


0.0564 (5.48%) 


Overall 


0.024 (2.41%) 


0.1038 (9.86%) 


In Percentage Change 


Private 


Public 


Permanent 


0.0081 (0.81%) 


0.0072 (0.72%) 
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Long-run 


0.0137(1.36%) 


0.0201 (1.99%) 


Medium-run 


0.0256 (2.52%) 


0.055 (5.37%) 


Short-run 


0.0791 (7.60%) 


0.064 (6.22%) 


Overall 


0.0396 (3.88%) 


0.055 (5.35%) 



In terms of K-12 enrollment, the results suggest that private 
schools exhibit bigger overall effect on Hawaii's income growth 
relative to that of public schools, confirming the previous result under 
the bi-variate feedback method. However, the feedback effect is 
concentrated mainly in the short-run (2-3 years) to medium-run (4-12 
years). Conversely, participation in K-12 public education exhibited a 
significant long-run to permanent effect on Hawaii's income growth 
relative to that of private education. This result may be explained by 
the growing number of high school graduates migrating out of the 
state. For example in 1992, the net out-migration of high school 
graduates was recorded to be around 690 and increased to 958 four 
years after. Apparently, families who could afford to send their 
children to private schools are willing to spend a little more to send 
them out of state in anticipation of more and better choices in 
education available in the mainland. 

In terms of tertiary level, the overall contribution of public 
school enrollment to Hawaii's income growth is bigger than that of 
private universities. However, decomposing the feedback effect by 
frequency suggest that this dominance of public enrollment in 
explaining the variation in income growth seem to be concentrated 
mainly in the short-run to medium-run. Conversely, private 
universities exhibit a permanent and long-run effect in explaining the 
variance in Hawaii's income growth relative to that of the public 
university. This finding might suggests that private tertiary education 
may be the key to promoting long-run growth in Hawaii. Similarly, 
one cannot ignore the significant contribution of the public university 
in building Hawaii's human capital stock in the short to medium-run. 

In terms of all educational levels, i.e. primary, secondary and 
tertiary level combined, participation in public education tend to 
explain a greater proportion of variance in Hawaii's income growth 
relative to private education across almost all frequency levels. Again, 
this finding confirms the previous result found in the bi-variate 
feedback method. 

Concluding Remarks 

In this study, an empirical investigation is done to assess and 
compare the relative contribution of public and private schools to 
Hawaii' economy. I employed the unrestricted vector autoregression 
(VAR) model that uses only past regularities and historical patterns in 
the data to examine the dynamic feedback relationship between 
participation in education and income growth. The results suggest that 
across all educational levels, i.e., K-12 and tertiary, participation in 
public education could be a good predictor of income growth in 
Hawaii. However, decomposing the feedback effect by frequency 
suggests that the dominance of public education in explaining the 
variation in income growth to be concentrated mainly on the short-run 
to medium-run for tertiary level and long-run to permanent effect for 
K-12 level. Hawaii state legislature and educators should perhaps take 
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these results as a motivation not to ignore the problems plaguing 
Hawaii's public schools but should work towards greater improvement 
and support for public education given its predicted significant overall 
contribution to the economy. Similarly, the presence of significant 
contribution of K-12 private schools in the short-run to medium-run 
and private universities' long-run to permanent effect on Hawaii's 
income growth should serve as a driving force that could help bring 
about healthy competition and greater efficiency in the provision of 
educational services in Hawaii. 
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Appendix 

Two Related Linear Dependence and Feedback Measures 

A. Geweke's (1982) method is used to measure the degree of 
dependence or the extent of various kinds of feedback between data 
series. He defined the measures of linear dependence between say, X 
and Y wide-sense stationary series in terms of the following linear 
projections, 

0)Yt = S"s=l“lsYt-s+^”s=l«2sXt-s + ^U 

(3)\ = n=,Y,3Y..3 + U3, 

where the linear feedback measure from X to Y is defined as 
Fx tvar (03^)/ var(Ujj)] 

while the measure of contemporaneous feedback between X and Y is 
defined as 



Fj^oy = log [var (u,j)/ var(u 2 j)]. 

So, the measure of linear dependence between X and Y or Fj^ Y is the 
sum of linear feedback from X to Y, F^^^y , linear feedback from Y to 
X, Fy_^j^ and instantaneous linear feedback F j^oy 

^X,Y ^X-^Y ^ Y->X ^ X“Y 

where Fy_^^ is found by switching X and Y in equations (1) and (3) 
and in the definition of directional feedback. 
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B. Building on Geweke's feedback measure, McGarvey(1985) 
developed a useful alternative summary measure by decomposing the 
feedback by frequency in order to distinguish between short-run and 
long-run effects of a given innovation or shock. 

In the context of this study, the MA representation of the 3- 
variable orthogonalized autoregressive system is as follows: 

rxjrcj,(L)c,2(L)c,3(L)iroj 
eYj = eC 2 i(L)C 22 (L)C 23 (L)| I coj 

LZjJLC 3 j(L)C 32 (L)C 33 (L)JLritJ 

where for example, C 2 j(L) gives the response of to 
innovations in X^. and, the overall feedback from X to Y is defined as 

Fx ^ C2J (s)^var(Uj)] 

The transformation ( l-exp[-Fj^ gives the proportion of Y's 
variance explained by shocks to X. 

To distinguish between short-run and long-run effects, the overall 
feedback is decomposed frequency bands. Feedback from X to Y over 
the interval (A,j , ^- 2 ) is defined as 

h ’ ^ 2 ) = log K Ixi / (Jj^j ^2 Sy(^) - C 2i(X) 2 a^2) 

dX)\ 

since var(Y) = (1/2 n) \^^S^{X)dk and SyCX) = C C 22 (>^) 

2 ct^2+ c 22 ^^ So, if Uj contributes nothing to the variance of Y 

at frequency A,, the ratio will be one and the feedback measure will be 
zero. Note that a period of a cycle is defined as the ratio of 2 tc to the 
frequency. 
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Abstract 

Over the past decade, several states have created 
comprehensive accountability systems designed to 
increase student learning in public schools. These 
accountability systems are based on "high-stakes" 
standardized testing of a state curriculum. Rewards and 
interventions for local educators are based largely upon 
students' performance on these tests. Using the recent 
accountability reforms in Georgia as a backdrop, this 
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article considers the role of local flexibility within such an 
accountability system— flexibility over paperwork, 
resources, personnel, and curriculum for local educators. 
Increased flexibility for local educators is not merely an 
option in a world where local educators are subject to a 
comprehensive accountability system imposed by a state— 
it is a requirement for success. We make a case for 
providing local flexibility and provides a discussion 
regarding types of flexibility, vehicles for granting 
flexibility, and who should receive flexibility. 



I. Introduction 

Over the past decade, several states have created comprehensive 
accountability systems designed to increase student learning in public 
schools. These accountability systems are based on "high-stakes" 
standardized testing of a state curriculum. Local educators face 
consequences based on how well their students do on these exams. 
They receive rewards for good student performance and are subject to 
interventions to rescue children from low-performing schools. Using 
the recent accountability reforms in Georgia as a backdrop, this article 
considers the role of local flexibility within an accountability system- 
flexibility over paperwork, resources, persoimel, and curriculum for 
local educators. 

This increased flexibility involves a decentralization of authority 
that is broader than, but inclusive of, school-based management. To 
implement the increased flexibility for local educators contemplated in 
this article would require profound decreases in the dizzying array of 
state and federal regulations governing local districts, schools, and 
personnel. 

Increased flexibility for local educators is not merely an option 
in a world where local educators are subject to a comprehensive 
accountability system imposed by a state— it is a requirement for 
success. Failure to provide local educators with flexibility to meet 
statewide learning goals for students would lead to blurred lines of 
accountability, and would not capitalize on the unique talents of local 
educators and other unique local circumstances, both of which would 
ultimately prevent accountability systems from realizing their foil 
potential. In addition, this failure will likely prevent accountability 
systems from surviving the political battles that periodically surround 
public education. This article makes a case for providing local 
flexibility within an accountability system and provides a discussion 
regarding the granting of flexibility, types of flexibility, vehicles for 
gremting flexibility, and who should receive flexibility. 

To those ends, section II provides an overview of 
comprehensive accountability systems that measure school and student 
performance and hold local educators accountable for this 
performance. Section III describes the appropriate role for local 
flexibility within such an accountability system. Types of local 
flexibility and vehicles for granting this flexibility are presented in 
sections IV and V. Section VI discusses who should get flexibility, 
including guiding principles for state policymakers. Section VII 
contains concluding remarks. 
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II. Overview of Accountability 

Several states, including Florida, Kentucky, North Carolina, 
and, most recently, Georgia, have created comprehensive 
accountability systems designed to increase student learning in public 
schools. A "comprehensive" accountability system has each of the 
following three components: 

1 . Goals for student learning at all grade levels. 

2. Accurate measurement of student learning outcomes. 

3. Rewards for local educators (superintendents, principals and 
other administrators, and teachers) for good student outcomes 
and interventions to rescue children from failing schools. 

We served on the staff of the Governor's Education Reform 
Study Commission (GERSC) that led to the creation of the 

comprehensive accountability system for Georgia's public schools.’ At 
the first Commission meeting in June 1999, Georgia Governor Roy E. 
Barnes announced the "charge" for GERSC: 

Let us come to the table and pool our best ideas, let us 
bring our best-hearted intentions, and let us steel up our 
best resolve to ensure for our children tomorrow a better 
system of public education than we find today. 

"Georgia Governor Roy E. Barnes 

In his speech. Governor Barnes announced the formation of four 

committees, the most ambitious of whieh was titled "Accountability."^ 
In describing the role of the Accountability Committee, the Governor 
said he wanted an end to "excuse-based education" in Georgia public 
schools. The committee members included business executives, 
legislators from both major political parties, retired educators active in 
education policy circles, and a retired college president (who is 

currently a professor).^ The Accountability Committee heard 
testimony from several local, regional, and national researchers and 
professional educators about accountability approaches in other states 
and important conceptual issues in designing an accountability system. 
Many of the individuals who made presentations were veterans of the 
standards and accountability movement and recommended 
comprehensive standardized testing on public school students in 

several subjects.^ The committee members were told that these exams 
should be curriculum-based exams called criterion-referenced tests, 
and these exams could be used as an important measuring stick for 
evaluating the performance of individual students, educators, schools, 
and districts. Using such exams for accountability purposes is 
commonly known as "high stakes testing." 

Although the Accountability Committee made no formal 
recommendations, the committee members coalesced around the 
following ideas: 

• Students in grades 3 and above should take curriculum-based 
exams at the end of each school year, based on the state of 
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Georgia's Quality Core Curriculum. 

• There should be threshold scores on these exams that indicate 
how well each student has mastered the material. 

• Local districts, schools, and educators should be held 
accountable for how well their students performed on these 
exams by a system of rewards for good performance and 
interventions for persistently low levels of student learning. 

• These rewards and interventions should be based on levels of 
student learning and improvements. 

• There should be a new, agency, independent of the state’s 
Department of Education, created to monitor performance of the 
exams and other educational outcomes and to administer 
rewards and interventions. 

House Bill 1187, passed by the Georgia General Assembly in 
the Spring of 2000 and signed into law by Governor Barnes set up a 
state accountability system for public education in Georgia that closely 
followed the thinking of the members of the Accountability 
Committee (Chapter 20 Section 14 of the Georgia Official Code). 

For the first time, Georgia had an accountability system based 
on the end product of education— student learning. The state 
Department of Education, which prior to the formation of the 
education commission had already begun refining the state curriculum 
(QCC) and creating curriculum-based exams, would administer 
statewide exams to students in grades 3 and above. These exams were 
first administered in some grades and subjects in the Spring of 2000, 
and they will be fully implemented in all grades and subjects by 2002. 
For both schools and educators, the law provides several rewards for 
good performance and interventions for persistently low performance- 
performance defined in terms of student learning of the state’s 
curriculum and other, currently unspecified, student outcomes. 

Rewards and interventions will be implemented once the exams are 
fully implemented. The rewards and interventions passed under the 

new accountability law include^: 1) Two grades (A through F) will be 
awarded to each school, where one grade will based on levels of 
student learning outcomes and the other grade will be based on 
improvements in student learning outcomes; 2) All certified personnel 
at a school will be given a $1,000 bonus for each "A" grade the school 
receives and a $500 bonus for each "B” grade. 

"If a school has received a grade of D or F for a period of two 
consecutive years or more, the State Board of Education could appoint 
a school master or management team to oversee and direct the duties 
of the principal of the school in relation to the school until school 
performance improves and the school is released from intervention by 
the director, with the cost of the master or management team to be paid 

by the state.’’^ 

"If a school has received a grade of D or F for a period of three 
consecutive years or more, the State Board of Education shall 
implement one or more of the following interventions or sanctions, in 
order of severity; 

• (A) Removal of school personnel on recommendation of the 
master or the school improvement team, including the principal 
and personnel whose performance has continued not to produce 
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student achievement gains over a three-year period as a 
condition for continued receipt of state funds for administration 

• (B) Allow for the implementation of a state charter school 
through the designation by the State Board of Education; 

• (C) Mandate the complete reconstitution of the school, removing 
all personnel, appointing a new principal, and hiring all new 
staff. Existing staff may reapply for employment at the newly 
reconstituted school but shall not be rehired if their performance 
regarding student achievement has been negative for the past 
three years; 

• (D) Mandate that the parents have the option to relocate the 
student to other public schools in the local school system to be 
chosen by the parents of the student with transportation costs 
borne by the system; or 

• (E) Mandate a monitor, master, or management team in the 
school that shall be paid by the district."^ 

Although the newly created Office of Education Accountability 
(OEA), independent of the state's Department of Education, has not 
yet determined what student learning outcomes will determine these 
school-level grades that will drive the rewards and interventions. 
Nevertheless, it is mandated in HB 1187 that student performance on 
the curriculum-based exams, both levels of performance and 
improvements, will be the main determinant of each school's grades. 

The cash bonuses to teachers and the potential interventions, 
especially public school choice and opening a state-funded charter 
school in the neighborhood of a failing school are not widespread 

across the U.S.^ 

Although the issues involved are extremely important, our 
purpose here is not to discuss the merits or demerits of accountability 
systems based on high stakes testing and state mandated rewards and 

interventions.'® This article considers the appropriate role of local 
flexibility within a comprehensive accountability system and several 
implementation issues, including what kind of flexibility to grant, how 
to grant it, and to whom should flexibility be granted. 

III. Flexibility Within An Accountability System 

In this section, a case is made for providing local educators and 
schools with flexibility-if and only if the local educators are held 

accountable for their performance by an outside entity. ' ' At the end of 
this section, we discuss the issues of negative unintended 
consequences that may be present in systems of comprehensive 
accountability and governance in a decentralized system. 

The argument for flexibility under a system of accountability 
has been made before (Hanushek, 1994; Haimaway, 1996). We 
reiterate their claims and discuss implementation and political 
economy concerns as well. 

Comprehensive accountability means three things: goals for 
student learning, student learning outcomes are accurately measured, 
and local educators (school boards, superintendents, principals and 
other administrators, and teachers) are subject to rewards for good 
student outcomes and interventions to rescue children from failing 
schools. If truly held accountable for student learning outcomes, local 
educators have strong incentives to do whatever it takes to achieve the 
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specified student learning goals. 

Without a significant degree of control over the means for 
education improvement, such as budgets, personnel, and curriculum, 
local educators cannot ultimately be held accountable for achieving the 
assigned end of improving and achieving a high level of student 
learning. This point is best demonstrated by considering two polar 
opposite forms of "accountability." 

Accountability over Inputs, Process, and Program Implementation 

Suppose the state were to give each school student learning 
targets and a prescription for how to achieve those targets. If a 
particular school obediently implemented the state's prescription and 
the level of student learning was low and was not improving, then this 
failure belongs to the state— not the school. This school would be 
responsible for properly implementing the state's prescription; the 
school would not be ultimately responsible for student learning. Local 
educators, who possess better information about their own unique 
talents and the local circumstances, including the types of students, 
resources, and environment, may wish to deviate from the centrally 
prescribed education formula because they deem such changes as 
beneficial in their local situation. Local educators who acted on those 
wishes would be subject to sanctions for not following the script they 
were given. 

This approach to "accountability" makes state policymakers 
responsible for student learning outcomes and local educators 
responsible for implementing the state's prescription. This blurs the 
lines of accountability, because lawmakers, parents, and other citizens 
will inevitably blame local educators for any low performance by 
students. In addition, the unique talents of local educators and 
circumstances of local schools are not exploited to best achieve the 
state's goals. Taken together, these two problems will frustrate local 
educators and could lead to poor implementation of state education 
policy, higher teacher and administrator attrition, and even a reversal 
of state education policy regarding standards and accountability as 
local educators make credible arguments that state policy inhibits local 
creativity and misplaces blame for any failures. 

(Results-Based) Comprehensive Accountability 

A second approach is to have state and local education 
authorities articulate the desired standards for student achievement and 
hold schools and educators accountable, through rewards and 
interventions, for meeting or failing to meet the standards. Choosing 
the second path, the path of accountability for student learning and 
flexibility on "how to" meet the goals, local educators are encouraged 
and empowered to pursue their own paths for success, and, thus, they 
would be ultimately responsible for the results of their own initiative. 

The purpose of flexibility within a results-based accountability 
system is to allow educators and schools to create their own roadmaps 
for educational success given their unique student populations, 
circumstances, and personnel. Although local school systems, schools, 
administrators, and teachers across the nation have varying degrees of 
flexibility, even when there is little or no local accountability, the level 
of flexibility that is desirable under a results-based accountability 
system is much larger than that which is desirable under the typical 
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notion of "accountability"-accountability over on inputs, process, and 

1 9 

program implementation. 

School-based management plans (flexibility plans) implemented 
in various states across the nation do not typically provide much local 
flexibility, and there is not any evidence that school-based 
management alone leads to better student outcomes (Summers and 
Johnson, 1996). In fact, an Education Commission of the States report 
cites a RAND study that concluded that no school-based management 
effort "has yet created the hoped-for dramatic improvements in school 
quality (Education Commission of the States, 2000)." 

State policymakers in Georgia, and in other states as well, are 
reluctant to give local educators more autonomy. This reluctance is 
understandable given the lack of true accountability in most states. In a 
world without results-based accountability, many education rules and 
regulations are absolutely necessary: Relative to a world with no 
accountability over process or results, such rules and regulations 
promote good student outcomes. The goal of rules and regulations is to 
elicit good school performance, students learning beyond expectations. 
However, most local educators believe that many of the current laws 
and rules, through unintended consequences, hinder them from 
offering the best possible education to each child. However, 
eliminating such rules is not necessarily warranted— local educators 
have little incentive to act in the best interest of children without the 
rules if they are not held accountable for student outcomes— it is 
human nature. This is not a characteristic of educators, but one of 
1 

humans in general. 

If given autonomy, what incentive does a local educator have to 
pursue whatever it takes to make sure that students achieve if they are 
not held accountable for student learning outcomes? If given 
autonomy over hiring decisions, what is to stop a principal from hiring 
a relative or friend who may not be competent as a teacher? Nothing 
unless there are certification rules and other regulations to drastically 
mitigate the chance of this occurring. (Of course, no set of rules can 
completely eliminate all corruption or well-intentioned, but misguided, 
policies). However, these rules limit the principals' flexibility over 
who he or she can hire to teach. These rules are desirable if the 
principal is not held responsible for student learning outcomes. If 
principals are held responsible, such rules may not be necessary, and 
may even harm student learning by denying some good teachers the 
ability to teach in our public schools. 

Given the reluctance of state lawmakers and officials to grant 
local flexibility, local educators, who desire more flexibility over 
resources, personnel, and curriculum, because they believe that their 
students will benefit from doing things in different ways, will have to 
accept results-based comprehensive accountability (student learning 
goals, measurement of progress toward the goals, and rewards for 
success and interventions to rescue children from persistent failure) in 
exchange for the increased flexibility. But there is another side to that 
coin: State policymakers who wish to impose results-based 
comprehensive accountability systems on local educators may have to 
grant increased flexibility to see their accountability reforms realize 
their full potential and to make their accountability reforms "stick" 
politically. Failure to judiciously increase flexibility may lead to a 
gradual erosion of accountability measures. If local educators who 
have little or no flexibility to improve schools are blamed for any 
school failures, then such a situation is not likely to be politically 
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tenable. Those wrongly blamed will make the arguments that they are 
held responsible for things beyond their control, and the end result 
could be the erosion of accountability based on student leaming--and 
any benefits that would come from such an incentive system. 

There are opportunities for obtaining flexibility available to 
local schools under current Georgia laws and regulations, and, by and 
large, local educators are not taking advantage of them. These 
opportunities include the waiver process, charter school conversions, 
and demonstration schools. Although there are many waiver petitions 
to the state's Department of Education to gain relief from state 
regulations, the vast majority of them are for the same two or three 
things (e.g. block scheduling). Charter school conversion opportunities 
have been available since 1993, and there have been less than 30 
conversions (out of 1,887 schools). The similar demonstration school 
process has been available since the mid-1980s, and, to our 
knowledge, there has been only one application. Georgia has recently 
begun implementation of a results-based accountability system, and 
this new era will likely result in a large increase in the interest of local 
educators in utilizing the existing flexibility to do things in different 
ways— because they are now held accountable for student learning. If 
local educators under a system of accountability do not wish to 
increase their autonomy over resources, personnel, and curriculum, 
then we suspect that the rewards for good performance and/or the 
consequences for failure are not providing strong incentives or 
motivation. That is, the accountability system would not be 
comprehensive because it does not contain adequate rewards and 
interventions. 

Negative Unintended Consequences and Increased Local 
Flexibility 

Many education policymakers and researchers have expressed 
concerns about negative unintended consequences that may result from 
inaccurate measurement of student and school performance— which 
may cause schools with low and high average student performance 
from being incorrectly sanctioned or rewarded; measuring only some 
student outcomes— which will likely lead to too much emphasis on 
what is easily measurable; and incentives— which may distort 
educators' efforts in unintended, undesirable ways, such as a decline in 
collegiality which decreases inter-teacher professional development. 
Hannaway (1996) suggests that parents, the actors in a child's 
education closest to the situation, can be empowered, via 
decentralization, to act as monitors of the education process to 
minimize the harm caused by any negative unintended consequences 
of incentives and flexibility (decentralization). Through local 
(individual) school councils or via some form of parental school 
choice, parents will be empowered to make their voices heard 
regarding the actions of individual schools. Under its 2000 
accountability reforms, Georgia created local school councils 
composed of two parents, two "businesspeople," two teachers, and the 
school principal— the majority is non-school employees. Legally, these 
school councils have almost no power, but it will be interesting to see 
the impact of their mere existence on school and district-level 

decision-making.*^ 

Governance 
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Under regimes of centralized decision-making in public 
education, citizens exert their influence by electing school board 
members and/or federal, state, and local lawmakers who in turn make 
decisions regarding school policy. Providing increased autonomy to 
un-elected local educators would disenfranchise parents and other 
citizens from a large degree of the education policymaking process. 
Therefore, it is likely that any increase in decentralized authority 
would not survive politically if parents and other taxpayers did not 
have some mechanism of exercising their political rights over their 
children's schools, or the schools they finance. As stated above, two 
ways to implement this mechanism include local school councils and 
enhanced school choice. Given this line of reasoning, one may include 
"parents and other citizens" in the list of local educators who receive 
increased autonomy under increased local flexibility. This issue is not 
necessarily one of giving parents and other citizens more direct 
decision-making authority; the issue is at what level do the citizens' 

representatives make education policy decisions.'^ 

Issues for State and Federal Policymakers 

The first issue facing policymakers is whether there is a 
comprehensive accountability system that is solid enough to 
contemplated large increases in local flexibility. If yes, then the second 
issue is whether existing, perhaps largely unused, flexibility under 
current laws and regulations is enough to empower local educators to 
make whatever changes are necessary to increase student learning, 
which makes local educators ultimately responsible for student 
learning. A third issue is whether parents and other citizens are 
empowered to participate in education decision-making and to monitor 
their local schools to minimize the harm caused by any negative 
unintended consequences. This next section discusses areas in which 
states may consider granting increased flexibility to local educators. 

IV. Types of Flexibility 

Under a comprehensive accountability system focused on 
student learning outcomes, state and federal laws and/or regulations 
can be repealed to provide local educators with flexibility over four 
broad areas: Reporting Requirements; Financial Resources; Human 
Resources; Curriculum. We briefly discuss decentralizing authority in 
order to increase local flexibility over these four areas, and then we list 
specific examples of increased flexibility that could be granted to local 
educators, where these examples come from the flexibility offered to 
public schools that currently operate under a high degree of 
accountability: charter schools. 

Reporting Requirements 

Local educators, in both district offices and individual schools, 
must complete a lot of reports for both the state and federal 
governments. This paperwork is typically in the form of reports that 
must be completed before and after the receipt of funds from federal 
and state education programs. The pre-funding reports are typically 
plans on how the particular pot of money would be spent, and the post- 
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funding reports tend to be assessments of how successful the particular 
program was at implementing the program—the program's effect on 
student learning is all too often amorphous or nonexistent in post- 
funding reporting. Thus, the purpose of this oversight is to ensure that 
the money is spent in ways the state or federal government deems best 
for the students. Under the new comprehensive accountability system 
in Georgia, the Office of Education accountability will conduct results- 
based assessments. These will be formal assessments and performance 
measurements for student learning such as standardized curriculum- 
based exams. Given that these assessments directly measure-student 
outcomes— what the current oversight measure indirectly, much of the 
current reporting requirements may be superfluous. 

Filling out paperwork is arduous for local schools and systems, 
especially for smaller school systems. One associate superintendent of 
a small school district who we spoke with said that he spends about 
30% of his time on filling out reports— time that he feels could be 
better spent on instructional and programmatic improvements. In 
addition, the time and resources previously devoted to filling out and 
monitoring paperwork could be used to train local educators to be 

better managers of their increased flexibility.'^ 

Reduced paperwork has practical flexibility benefits as well. For 
example, some school districts have directors of technology. These 
directors must fill out a lot of paperwork on how state technology 
monies are spent. Any time these directors spend filling out paperwork 
is time not spent training teachers how to use the technology. 

We list two alternative ways for the state to reduce reporting 
requirements on local educators: 

1 . Have state education departments satisfy much of the reporting 
requirements imposed by the federal government; under 
accountability reporting requirements, state departments of 
education have the information necessary to fill out much of this 
paperwork. If the state assumed this reporting burden, local 
educators— the educators closest to the students— would have 
more time to focus on teaching and learning. 

2. Give local schools systems and schools more flexibility over 
financial resources. Having flexibility over financial resources 
would allow local educators to spend less time reporting (to the 
state) how each dollar is spent. 

Financial Resources 

Many public school districts receive monies from states through 
foundation grants and categorical grants. Individual public schools, in 
turn, receive monies from school systems. Superintendents and 
principals could be empowered to spend more state monies in the ways 
they deem most appropriate to best educate their unique student 
populations. In addition, any state regulations, explicit or implicit, of 
local money could be repealed as well. The purpose of flexibility over 
financial resources is to empower those closest to the children to try 
new things, to augment existing programs that are working, and reduce 
or eliminate programs that are not working for their particular students 
(such programs may work in other places for idiosyncratic reasons). A 
by-product of this flexibility would be to reduce paperwork for system 
and school administrators and teachers, which would allow them to 
spend more time focusing on doing whatever it takes to improve their 
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schools. 

A good way to demonstrate flexibility over financial resources is 
through an extended example. Individual school districts in Georgia 
typically get English to Speakers of Other Languages (ESOL) dollars 
from the state. Many small school districts have only a small number 
of ESOL students, so the state money they receive for the ESOL 
program does not cover a full teacher’s salary. School districts that 
have such scale issues must use locally generated funds or other state 
funds for personnel to pay the balance of an ESOL teacher's salary. In 
addition, the district would have to use local funds for ESOL 
materials. Authority to use other state funds to purchase ESOL 
materials would free up the local money for other programs that local 
educators deem most important given their particular student 
populations, faculty, staff, and environment. 

Where would "other state funds" come from? Wouldn't those 
other state funds be better spent on the programs for which they were 
earmarked by the state? Perhaps, but consider an additional scenario. 
School districts in Georgia often receive money from the state based 
on the system's "needs," needs as determined by the state, and these 
needs tend to be drive by overall FTE counts and FTE counts for 
various student sub-groups that are calculated to the hundredth 
decimal place. For example, a system may receive state funding for 
2.35 guidance counselors. Per state regulations, the system that 
received funding for 2.35 guidance counselors must hire two guidance 
counselors. Under Georgia law, the remaining 35% of a guidance 
counselor must be spent on guidance counselors or direct teaching 
personnel, or else the money reverts back to the state. That is, the local 
system must use all that state money for guidance counselors or direct 
teaching personnel or lose it. Allowing local schools and systems to 
use the guidance counselor money for guidance counselors or direct 
teaching personnel is an example of flexibility over resources available 
under current laws and regulations. However, there could be increased 
flexibility over that state taxpayer money. Suppose local educators 
believe that because of their superior guidance counselors or students 
(of for any other idiosyncratic reason) that the money that was 
originally intended to hire 35% of a guidance counselor does not need 
to be spent on guidance counseling or direct teaching personnel. 
Suppose the school system would rather use those state funds for a 
competing, albeit worthy, program. Suppose the school system wants 
to use those funds for ESOL materials. Under current Georgia law, this 
money must be spent on hiring a third guidance counselor or part of a 
teacher, and this requirement may not lead to the highest and best use 
of those funds. 

1 7 

Another good example is maximum class size restrictions. 
Without a comprehensive accountability system, such class size 
restrictions may be necessary to ensure that the state money is spent 
wisely. However, under incentives from a comprehensive 
accountability system, can states trust local educators to spend that 
money wisely? Are smaller classes always the best use of those funds? 

Alternatives to give local educators flexibility over financial 
resources include having fewer state programs and give the monies 
formerly earmarked for programs to local schools on a foundation 
basis, and allowing local educators to spend monies earmarked for less 
than 50% of a position in any ways they deem necessary. 

Human Resources 
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Regarding what types of individuals may be hired for some 
tasks and how much individuals are paid, systems and individual 
schools in Georgia are bound by three major state laws: teacher 
certification, "fair dismissal" (tenure), and the salary schedule. 

The purpose of certification is to ensure that only individuals of 
a sufficient competency are permitted to be teachers. Under 
certification laws and regulations, sufficient competency of potential 
teachers is determined centrally, not by local systems and schools. 

This is in contrast to higher education and private K-12 education 
where potential teachers are evaluated by individual schools and 
departments within schools. An unintended consequence of 
certification requirements is that some prospective teachers feel that 
they face too large of a barrier to offering their services to schools. 
How many prospective educators are deterred is unknown. 

Individual schools and systems could be granted flexibility over 
who is permitted to teach. The elimination of certification 
requirements, including alternative certification requirements, would 
open the doors to teaching to individuals who are not willing to go 
through the process of obtaining certification. For example, programs 
such as Teach for America carefully screen recent college graduates 
and place them as teachers in schools. These new college graduates 
typically did not study education, and many of them wish to teach for 
only a short period of time. Local systems and schools could be 
empowered to decide for themselves if they wish to screen new 
college graduates, older folks looking for second careers, or others 
who are not certified to see if any or many of them would make good 
teachers. Thus, the issue is not one of teaching quality; it is a question 
of who decides whether an individual is competent to teach. In 
addition, states could recruit and screen exceptional college graduates 
who did not study education as undergraduates and any others 
interested and market these potential teachers to local systems and 
schools. Such a state program would provide local educators with 

1 R 

flexibility by expanding the pool of possible teachers. 

Flexibility over "fair dismissal" (tenure) provisions could be 
granted to local schools or school systems— individual school systems 
or schools could be granted the authority to design their own "fair 
dismissal" policies. Held accountable for results, individual systems 
and schools would have the incentive to create fair dismissal policies 
that allow them to maximize student learning. Georgia decentralized 
its statewide tenure provisions, by severely weakening them, as part of 
its 2000 education reform law. Only the Clayton County school 
district, so far, has created its own tenure provisions that provide more 
protections than the new state law. Nevertheless, these provisions are 
weaker than the previous state fair dismissal protections (Sansbury, 
2000 ). 

Many southern states have a minimum teacher salary schedule 
that is based on years of service and training. Elimination of a state- 
mandated salary schedule for teachers would allow individual schools 
or systems to decide whether they want to pay less to some teachers so 
that they may pay more to teachers they deem as important 
contributors to the overall mission of the school. Used judiciously, 
such policies could help schools retain good teachers, and provide 
incentives for bad teachers to find something else to do. An additional 
form of flexibility would be to allow for schools to lure teachers by 
offering to start them at a higher step on the schedule than their years 
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of service and training would dictate. This flexibility would allow 
schools to pay more to better teachers, which may enhance retention of 
these superior teachers. 

Curriculum 

In the new standardized testing in Georgia, students will take 
exams designed to test Georgia's Quality Core Curriculum (QCC). The 
QCC is designed to be the minimum amount that students should learn 
in the various grade levels and subjects. Local educators could remain 
free to augment the QCC in new and creative ways. Of course, 
mandating statewide curriculum-based testing severely restricts local 
educators' autonomy over curricular decisions. Therefore, states are 
responsible for the quality of the curriculum, and, under flexibility, 
local educators are responsible for whether students' master the 
curriculum. 

What specific flexibility ought to be granted? 

One level of flexibility is the current level of state and federal 
regulation applied to private schools, which is minimal. A less extreme 
level of local flexibility is the flexibility requested or the flexibility 
actually granted to "traditional" and "conversion" charter schools. 
Traditional charter schools that are public schools that are not 
neighborhood public schools; traditional charter schools are schools of 
choice. Traditional charter schools face a stronger set of incentives 
than other public schools; two outside actors hold them accountable 
for results, a central authority and parents. Traditional charter schools 
are, in theory, able to gain a large degree of autonomy in exchange for 
the possibility of a death sentence— if the charter school does not meet 
performance goals specified in their charter, a local school district or a 
state may revoke the charter, which means the school closes. In 
addition, traditional charter schools are held accountable by parents 
who may or may not decide to enroll their child in the charter school. 
Where traditional charter schools exist, parents have the option of 
sending their child to their neighborhood public school or the charter 
school. Typically, before central authorities have closed failing charter 
schools, there have been dramatic drops in student enrollments at these 
schools. 

Given these strong incentives to provide a high quality 
education to its students, traditional charter schools have the incentive 
* to seek to free itself from any rules and regulations that hinder 

teaching and learning. Therefore, any state or federal entity that seeks 
to identify any rules and regulations that may hinder teaching and 
learning in neighborhood public schools should look to rules and 
regulations that charter schools seek to escape, and the rules they 
actually escape. 

Conversion charter schools are neighborhood public schools that 
have received increased flexibility from the state. These schools are 
different than traditional charter schools in that they are not schools of 
choice. According to officials in the Georgia Department of Education, 
conversion charter schools ask for, and receive, far less local 
autonomy than traditional charter schools. Conversions have 
maintained most of their previous organizational structure and 
curricular goals, but asked to be exempt from such things as report 
cards regulations, how they handled certain categorical funds, and 
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when they tested their students. It is likely that these schools could not 
obtain even more flexibility, and do not seek to do so, because they are 
not schools of choice. That is, they do not operate under the strong 
accountability faced by traditional charter schools. Perhaps the change 
in school governance in Georgia via school councils will provide the 
accountability and citizen authority necessary for conversion charter 
schools to ask for and receive increased flexibility. 

For the research reported here, we interviewed several 
individuals about their experiences with flexibility and the flexibility 
given to charter schools: 

• Beverly Shrenger, Coordinator, Georgia Charter Schools, 
Georgia Department of Education; 

• Deborah McGriff, Edison Schools, Inc.; 

• Rich O'Neill, Edison Schools, Inc.; 

• Greg Giornelli, Principal, Drew Elementary School, a traditional 
charter school in Atlanta, GA; 

• Regina Merriweather, Principal, Druid Hills High School, a 
conversion charter school in DeKalb County, GA; 

• Jeffrey Williams, Georgia School Superintendent's Association; 

• Paul Hill, University of Washington, RAND Corporation. 

Based on telephone interviews, we compiled a list of rules of 
regulations that charters typically seek to avoid. Any central authority 
considering whether to abolish rules and regulations facing local 
educators should look to the relief given to charter schools because, 
imder comprehensive accountability for results, all neighborhood 
public schools will have at least one important characteristic of charter 
schools—responsibility for student outcomes and consequences based 
on those outcomes. 

1 . Salary Schedules. Many charters want relief from salary 
schedules in order to have the capability to pay what Edison 
Schools, Inc. refers to as "comparable and competitive" salaries. 
These salaries are made up of an hourly rate, a yearly percentage 
increase, incentives and bonuses, and stock options. 

2. Curriculum. Schools want the ability to develop the criteria for 
their own lesson plans. Some charters use such prescribed 
curriculums as Core Knowledge while others are totally 
innovative and use curriculums particular to that school. For 
example, Edison schools prefers to use its own curriculum for at 
least 70% of class times, and the state or district can dictate the 
remaining 30%. 

3. Non-Categorical Use of Funds. Traditional charters typically 
receive complete freedom over their budget allocations at the 
school site. Conversion schools typically ask for only limited 
flexibility or one-time flexibility. For example, conversion 
charter schools may ask to use some funds ear-marked for extra- 
curricular activities to buy technology. 

4. External Reporting. The type and amount of process reporting to 
chartering agencies, school boards, and the Georgia State 
Department of Education is often less than what is required of 
traditional public schools. 

5. Grading. Many charter schools want to have the ability to 
deviate from traditional grading scales. Druid Hills Charter, a 
conversion, changed its grading scale so that the letter grade "D" 
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was inclusive of the 60 to 69 percentile. Some schools want 
to implement a policy of no grading, checklist reports, or even 
rely strictly on portfolio's to show students achievement. 

6. Seat Time and Scheduling. Charters have asked to be exempt 
from the states requirement of 150 hours of clock time per year. 
Edison Schools have a longer school year than most public 
schools, while some charters opt for longer school days. This 
coincides with the request to alter the daily schedule for students 
(i.e. block schedule) that require different time configurations 
than most districts currently operate under. 

7. Textbooks. Since many charters wish to fully implement their 
school design, they request the ability to choose textbooks that 
may or may not be approved by the local school board. 

8. Certification. Teacher certification has not been a large issue for 
many charters thus far, as most charters have hired primarily 
certified teachers. Charters do exercise their ability to hire non- 
certified teachers in hard to fill subjects such as math, science, 
and world language. Additionally, some charters allow teachers 

certified for grades k-3 to teach 4*^^ grade, for example. 

9. Promotion and Retention. Charters want the opportunity to 
choose which students are promoted and retained each school 
year. Charters feel that this exemption is imperative if they are 
going to be held accountable for each student's eventual success 
or failure. 

1 0. Assessment instruments. Some charter schools like to perform 
their own assessments, and request waivers from assessments, 
such as norm referenced testing, that are notused for 
accountability purposes. 

1 1 . Technology. Charter schools like to use technology in a way that 
is consistent with their instructional goals. According to the U.S. 
Department of Education, 96% of charter school classrooms 
nationwide were equipped with computers. However, charters 
like the capability of choosing their own software, the amount of 
time each student uses a computer and the ability to buy 
computers with multi-media capabilities. 

12. Service Providers. Charter schools are typically allowed to 
choose what non-educational (maintenance, janitorial, insurance, 
purchasing, legal, health, social, before/after school, 
transportation, athletic, etc.) services are offered and who will be 
the provider of those services. More than two thirds of charter 
schools nationally either provided the service themselves or 
used outside providers. 

Suppose a state goveriunent decides to provide local educators 
with flexibility over at least some of these areas. This local flexibility 
raises a governance issues regarding consent of the governed. Parents 
and other citizens, through their influence on the political process, may 
permit schools of choice more latitude over their resources, personnel, 
and curriculum because no parents are forced to send their children 
there. Since traditional charter schools are schools of choice, all 
parents who choose one for their children have revealed they support 
what the school is doing differently than the neighborhood public 
school. This argument would suggest that flexibility given to 
conversion charter schools is the appropriate amount of flexibility to 
provide, in exchange for accountability. However, under Georgia's 
new accountability law, individual school councils were created. 
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Although these councils do not currently have much authority, they 
would surely be megaphones for parents’ ^lnd other citizens' voices to 
be heard in school-level discussions of how to best use any increased 
flexibility. Therefore, under the stronger consequences facing local 
educators under Georgia's new accountability reforms and the presence 
of the local school councils, the wider latitude given to traditional 
charter schools is possible for neighborhood public schools. Another 
mechanism to solve the governance problem would be some form of 
public school choice. 

V. What are the vehicle(s) for granting flexibility? 

Once a state, or federal, government decides to grant increased 
local flexibility because of heightened accountability for local 
educators, how can it complete that task? Alternative vehicles for 
granting flexibility include: 

1 . An entity that analyzes each and every state, or federal, rule and 
regulation and decides which ones are not needed, and which 
ones may be abolished without changing a law. This entity 
would be analogous to then Vice-President Gore's National 
Performance Review that was created in 1993. This new entity, 
or a piece of an existing entity, could be charged with analyzing 
each and every state (federal) regulation of local systems and 
schools. Regulations deemed to be impediments to teaching and 
learning would be eliminated by the entity. 

2. A legislature and executive could analyze each and every state 
law and decide which are no longer needed under a 
comprehensive accountability system. Only the legislature and 
executive can change existing state (federal) laws. The 
legislature could devote some portion of a legislative term to 
reviewing existing laws regarding education and deciding which 
laws are antiquated given an environment of results-based 
accountability. Perhaps a one-time bipartisan committee could 
be formed to begin the task. This vehicle for granting flexibility 

was used in Texas and Florida. * ^ 

3. A permanent entity that has the sole responsibility of hearing 
petitions from individual schools and decides whether to grant a 
large degree of autonomy to individual schools in exchange for a 
promise of increased student leeiming beyond normal 
expectations. This is similar to the Georgia waiver process. The 
difference is that flexibility will be granted for a whole range of 
items at one time, in exchange for tangible, measurable promises 
of increased student learning. A permanent new entity, or piece 
of an existing entity, whose sole mission is to hear petitions 
from local systems and schools for large degrees of flexibility in 
exchange for accountability would provide a permanent vehicle 
for enhancing flexibility and accountability. Creating an entity 
that has hearing these petitions as its sole mission would 
expedite the waiver process, and one of its goals would be to 
become less arduous than the current waiver process. 
Agreements between this entity and individual schools or 
systems would be akin to performance contracts. Failure by the 
local educators to live up to the increases in student learning 
specified in the agreement could result in the loss of the 
flexibility. Any significant regression in student learning after 
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receiving the new flexibility could result in a state-mandated 
intervention, which would mean less local autonomy than was 
initially present. 

VI. Who Gets Flexibility? 

Under comprehensive accountability based on student learning 
(results), flexibility could be granted in three ways: as a feasible 
alternative, in a world of accountability based on results, to empower 
all local principals and teachers find their own roadmaps for success 
given their unique student populations, circumstances, and personnel; 
as a reward to a school or system for high levels and/or improvement 
in student learning; as an opportunity to low performing schools to 
improve. 

Thus, differing degrees of flexibility may be granted all schools, 
only schools that demonstrate a high level of performance, and/or only 
schools that demonstrate a low level of performance. Texas, for 
example, dramatically reduced the regulations that the Texas 
Education Agency imposed on local systems and schools. After the 
passage of their accountability law in 1993, the Texas House and 
Senate Education Committees met jointly to eliminate all state laws 
and policies that addressed "how" local educators should provide 
schooling to children. At that time, the chairs of each committee were 
from different political parties, and the "scrubbing" of their state laws 
and policies went very well and was bipartisan. The flexibility law that 
subsequently passed stated that local systems had to abide by the 
accountability code and the funding code. The law also contained 
specific language to prevent the Texas Education Agency from making 
any policies that did not pertain to the accountability or funding 
portions of the Texas state code. The relationship between the state 
and local school systems in Texas is one of "if it is not in the 
accountability or funding code, then you can do it. No questions 
asked." Examples of regulations that were eliminated in Texas include: 
the length of the school day and year, seat time for specific subjects, 
and the minimum required number of library books per pupil. 

Texas has not yet been able to document to what extent the 
increase in flexibility led to its recent increases in student 
achievement. (We obtained all information about Texas from a phone 
interview with Dr. Criss Cloudt, Associate Commissioner, Office of 
Policy, Planning, and Research, Texas Education Agency. Dr. Cloudt 
made a presentation to GERSC 1999.) 

Schools who demonstrate a high level of performance and/or 
improvement have demonstrated that they have "what it takes" to 
manage a school under the current rules and regulations. Such 
successful leadership could be entrusted with even greater flexibility, 
to see if they could increase school performance even higher. 

However, one could argue, "Why rock the boat?" if the school is 
already high performing. We want to "rock the boat" because we 
suspect that added flexibility, under accountability, will allow high 
achieving schools to do even better. Texas provides additional 
flexibility to schools that receive an "exemplary" rating from the state. 

Schools whose students are persistently low performing may 
credibly suggest that state laws and regulations are due part of the 
blame for this low performance. Some suggest that these schools 
should be given added flexibility, above what is given to other schools, 
in order to see if they can improve. Others argue that giving these 
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schools added flexibility would reward failure. 

Guiding Principles of Granting Flexibility 

Based on the arguments made here, we offer three guiding 
principles for any state or federal policymakers deciding whether to 
grant flexibility to local schools, systems, and educators: 

1 . Keep your eyes on the prize. The purpose of flexibility is to 
allow educators to better organize their systems, schools, and 
personnel in order to increase student learning. 

2. Trust but verify. Flexibility should only be granted in exchange 
for accountability, a promise that student learning would 
increase beyond normal expectations. Failure to meet the terms 
of the promise should result in loss of flexibility. If under the 
flexibility, student learning in the school significantly regresses, 
the school should receive help, which would leave the school 
with less flexibility than it had initially. Under this principle, 
flexibility could be given to all schools, including low 
performers. 

3. Remove existing barriers to creativity that strives for excellence. 
Any system or school that wants to improve should be allowed 
to try, in exchange for accountability for results. 

VII. Conclusion 

With the passage of HB 1187 in the year 2000, Georgia's 
educational system has entered a new era of comprehensive 
accountability-results-based accountability. The state will set 
expectations and measure student learning outcomes; systems, schools, 
and local educators will be rewarded for exceeding the standards; and 
the state will intervene to rescue children from schools that are 
persistently falling below the standards. Given the movement toward 
comprehensive, results-based, accountability systems in several states, 
it is time to revisit the issue of decentralization in public education. 
Education researchers and policymakers should carefully consider the 
issue of decentralization within such accountability systems. To the 
extent that the rewards for success and interventions to rescue children 
from low performing schools prove to be significant in state 
accountability systems, local educators, in these states and in other 
states implementing similar incentives, should be given increased 
flexibility over paperwork, resources, personnel, and curriculum. 

The purpose of flexibility within a results-based accountability 
system is to allow educators and schools to create their own roadmaps 
for educational success given their unique student populations, 
circumstances, and personnel. The level of flexibility that is desirable 
under a results-based accountability system is much larger than that 
which is desirable under the typical way of doing things, which in 
Georgia pre Y2K reforms meant accountability based on inputs, 
process, and program implementation. Within a results-based 
accountability system educators and schools have strong incentives to 
do whatever it takes to achieve the specified student learning goals. 

Any increase in flexibility is only possible because of the new 
era of accountability. The more that systems, schools, and personnel 
are rewarded for successes and subject to interventions for any 
failures, the more flexibility that may be granted to local educators. 
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The combination of empowerment through local flexibility and 
consequences through rewards and interventions would give local 
educators the motivation and incentives to do whatever it takes to 
make sure the students in their care succeed. 

State-imposed comprehensive accountability systems are 
dramatic increases in state regulation of curriculum and assessment 
coupled with incentives for increased student learning. By themselves, 
these new state regulations may lead to better student outcomes, just as 
many centrally imposed regulations perhaps lead to better outcomes— 
in the absence of accountability based on student outcomes. 
Nevertheless, without increased flexibility for local educators— 
flexibility over paperwork, resources, and personnel— these standards 
and accountability reforms will not achieve their full potential because 
they fail to capitalize on the initiative and industry of local educators 
and they better information they have about unique local talents and 
circumstances. In addition, failure to empower local educators while 
holding them ultimately responsible for student learning outcomes will 
lead to an exodus from public education among local educators at best 
and political pressure from within to emasculate accountability 
reforms. For these reasons, we recommend large increases in 
autonomy for local educators under comprehensive accountability 
systems. 

Potential negative unintended consequences from high stakes 
testing, incentives, or local flexibility can be mitigated by school 
governance changes such as school councils and school choice. 

Agenda for Policy Research 

As noted by Hanushek (1995) and Hannaway (1996), how well 
students learn under comprehensive accountability systems coupled 
with local flexibility will determine the success of this, and any, 
education reform. In particular, a system of incentives and/or local 
flexibility may cause unintended harm to student learning by focusing 
too much attention on what is easily measurable and/or allowing 
autonomy to be misused without responsibility. Therefore, education 
research must increase efforts to evaluate the effects of these reforms 
on student learning. In addition, given that the basis for increasing 
local flexibility is premised on accurate measurement of student 
outcomes and significant incentives through rewards and 
interventions, education research must also analyze the extent to which 
measurement of student outcomes is accurate and incentives are 
meaningful. Without these conditions present, states and the federal 
government should be wary of going down the path, despite its 
promise, of increased flexibility for local educators. 

Notes 

' Scafidi and DeJamett served on the staff of GERSC in 1999 and 
2000, and Freeman served on the GERSC staff in 2000. 

^ The full text of this speech is available on-line at 
http;//ganet.org/governor/edreform/speech.html 

The other three committees were: 
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Funding: which studied ways to increase equity of school funding 
across the state. 

School Climate: which, in the wake of the Columbine (Colorado) and 
Heritage High (Georgia) violence, studied ways to make schools safer. 

Seamless Education: which studied ways to make the transitions from 
k-12 to technical colleges, and k-12 to two-year colleges easier. 

^None of the accountability members were officials in teachers' unions 
or professional organizations or any of the other professional 
organizations for educators (school board, school superintendents, 
etc.). 

^For example, Georgia State University Professor Gary Henry made 
several presentations to the accountability committee very early in the 
process. Previously, he served as an education official in Virginia and 
consulted with the state of North Carolina when they set up a testing 
and accountability system. Gwinnett County (GA) School 
Superintendent Alvin Wilbanks, who instituted high stakes testing in 
his district's schools, served as a Committee member as well. These 
individuals, as well as education officials from Texas and North 
Carolina who also made presentations to the accountability committee, 
seemed to support very strongly the "Standards and Expectations" 
recommendation of the 1 983 "A Nation at Risk" report, which called 
for "rigorous and measurable standards, and higher expectations, for 
academic performance and student conduct." 

^Many states that have implemented accountability systems such as 
Georgia's have rewards and interventions. For example, Kentucky and 
North Carolina give bonuses to teachers at schools deemed successful 
(based on their students' outcomes). Florida allows students at schools 
deemed failing to transfer to other public schools in the district, at the 
district's expense and offers them a modest scholarship to attend a 
private school (the private school portion of thes school choice option 
is currently under litigation). 

^Georgia Official Code 20-14-14 

^Georgia Official Code 12-14-41 

^Examples of these consequences around the nation include cash 
bonuses to teachers in schools deemed successful in Kentucky and 
North Carolina and public school choice for students in schools 
deemed persistently failing in Florida. For schools deemed failing, 
many states have reconstitution and removal of school persormel at the 
behest of the state, but these provisions are rarely implemented. 

Failure to utilize interventions leads to an elimination of any 
motivational benefits that come from having consequences. 

'*^Helen Ladd has wondered, "whether the undesirable side effects of 
accountability and incentive systems can be kept to a tolerable level 
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(Ladd, 1996)." Issues surrounding standards and accountability include 
the accuracy and quality of curriculum-based exams, whether such 
exams and incentives in general cause too many unintended 
consequences that harm teaching and learning, and whether all 
children benefit from these exams and from comprehensive 
accountability systems. Good references on these debates over 
accountability and the validity of high stakes testing include, Behn 
(1997), Betts (1998), Hannaway (996), Kohn (1999), Ladd (1999), 
Koretz, et al (1996), Grissmer, et al (2000), Klein, et al (2000), and the 
contents of the volumes Holding Schools Accountable (1996), edited 
by Ladd, and Improving America's Schools: The Role of Incentives, 
edited by Hanushek and Jorgensen. 

* ^Thus, federal government efforts to provide local educators more 
flexibility over federal taxpayer resources could be made contingent 
upon the existence of comprehensive state-level accountability 
systems. 

While working for GERSC, we heard several Georgia state 
employees suggest that it was "impossible" to have local flexibility 
and accountability at the same time. Their comments suggest that the 
culture of state monitoring of local inputs, process, and program 
implementation is alive and well— despite the passage of a results- 
based comprehensive accountability system the previous year. 
Examples of "accountability" for inputs, processes, and 
implementation include state regulations on maximum class size, state 
prescribed curricular programs, and teacher salary schedules. For each 
of these examples, many states hold local educators "accountable" for 
adhering to and implementing these state mandates. These examples 
are in contrast to states holding local educators accountable for results- 
-student learning outcomes. 

'^Surely college professors would love to avoid annual reviews of 
their research productivity, student evaluations of their teaching, and 
deans watching the number of students signing up for their courses and 
the national rankings of their departments. Each of these output 
measures is used to determine armual pay increases. In the absence of 
these measures of productivity, strong rules and regulations would be 
needed to govern professors' activities. Similar analogies can be made 
for all occupations, inside and outside of education. 

^ ^Chicago public schools have school councils that have real power- 
each school council hires its school's principal. See Bryk, et al (1998) 
for more details of the school governance structure in Chicago. Given 
the vast differences in authority, it is likely that the impact of Georgia's 
school councils will be much smaller, under existing legislation, 
relative to the Chicago experience. 

'^Under a school choice regime principals and teachers would be the 
local educators who directly make educational decisions, while parents 
vote to give local educators power to make decisions for their children 
through their school choice decisions. 

*^Hannaway (1996) suggests that local educators who are given, for 
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the first time, a large degree of autonomy need training in "the range 
of production possibilities in education" as well. If empowered to 
make changes, held sufficiently responsible for their decisions, and 
skilled in management, local educators will take it upon themselves to 
discover and create "possibilities in education." 

'^In its 2000 education reform law, Georgia began to enforce 
decreased maximum class sizes. 

*^The Massachusetts Institute for New Teachers (MINT) is such a 
state program. MINT recruits college graduates to enter its summer 
program. MINT students teach summer school in the morning and take 
teacher preparation courses in the evening. Individual public schools in 
Massachusetts are permitted to hire MINT graduates and grant them 
full certification. By expanding the pool of available teachers, the state 
has given local educators more flexibility over personnel (Scafidi, 
2000 ). 

'^Interview with Texas education official Dr. Criss Cloudt and 
documents from the Florida Department of Education and Florida 
Senate Bill 1770. 
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Abstract 

Despite more than a decade of research on bottom- 
up school chtmge, the principal/ superintendent 
relationship continues to be studied primarily as a 
traditional flow of power from the top down. There is 
little research that considers the proposition that power 
vested in principals can be exercised upwardly within the 
school district hierarchy in the form of independence from 
and influence on the superintendent. Given the lack of 
research on these phenomena, it is not surprising that we 
could find no studies that explore the effects of 
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hierarchical independence and influence on school 
climate. The present study investigates both. Two schools 
form the basis of this comparative case study. The schools 
were chosen based on scores obtained through the OCDQ 
and TAI instruments. The first school is selected for its 
high scores on both instruments and the second school is 
selected based on average scores on the OCDQ and the 
TAI. Both schools are in the same school district and a 
brief description of that district begins the discussion. 
Individual case study findings as well as a comparison of 
the two case studies follow. 



I. Hierarchical Independence and Influence 

According to Hoy and Miskel (1991), principals use hierarchical 
independence and influence within the formal structure of the district 
to give, various resources for the school. Hoy and Miskel define 
hierarchical independence as, "the extent to which administrators 
demonstrate their autonomy from superiors" (p. 81). To illustrate, a 
principal exerts hierarchical independence from the superintendent 
when she decides to implement a major reform or instructional 
iimovation, relying on her own expertise, knowledge, and ability to 
acquire resources rather than relying on guidance and resources from 
the superintendent or central office administrators (Fullah, Anderson, 

& Newton, 1986; Leithwood, 1988). 

Hoy and Miskel (1991) describe hierarchical influence as the 
ability of the principal to gain positive benefits for the school from the 
superintendent. Adapting from the above illustration, hierarchical 
influence is used when a principal persuades the superintendent to 
support a unique school program or to provide additional resources to 
the school 

Hierarchical independence and influence, though important, can 
be difficult for a principal to exercise. As middle level a administrator 
in a hierarchical organization, a principal simultaneously occupies a 
subordinate and superordinate position. Consequently, a principal must 
balance often-competing demands and expectations from the 
superintendent and teachers. Moreover, while both the superintendent 
and teachers value independent and influential action on the part of the 
principal, they do so for different reasons. 

For the superintendent, independence and influence are valued 
when problems are resolved at the school level of when external 
resources are secured with little encumbrance to the district (Crowson 
& Morris, 1984). Independent or influential action that is inconsistent 
with the expectations of the superintendent or that creates problems for 
central office is viewed with a less sanguine eve. Teachers, on the 
other hand, value independent and influential actions when these 
actions bring needed resources co the school, are consistent with the 
values held by the faculty (Porter & Lemon, 1988), or buffer the 
faculty from external demands and pressures 

School Climate 

We propose that teachers, as prime beneficiaries of a principal's 
upward exercise of power, are uniquely positioned to observe the 
effects of a principal's use of hierarchical independence and influence. 
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As teachers observe this dynamic, according to Boyan (1988), their 
perceptions of school climate are affected. Although there is little 
consensus concerning forces that mold school climate, variables that 
have been studied include principal leadership (Kottcamp, Mulhem, & 
Hoy, 1987; Purkey & Smith, 1983; Taylor& Tashakkori, 1995), 
teacher morale (Pallas, 1988), and academic emphasis (Hoy & 
Woolfolk, 1993). Some researchers (Halpin, 1966; Hoy, Tarter, & 
Bliss, 1990) describe school climate as ranging from open to closed. 
Schools with an open climate operate with few rules or regulations 
while schools with a closed climate are hampered with restrictive rules 
and regulations and close supervision (Hoy et al., 1990). 

These studies and others (e.g., Boyan, 1988) suggest that school 
climate is a mediating factor in the academic achievement of students, 
adding to its importance as a focus of educational research. As noted, 
we base the definitions of our terms on Hoy and Miskel (1991). These 
authors defined school climate as the "relatively enduring quality of 
the school environment that is experienced by participants, affects 
their behavior, and is based on their collective perceptions of behavior 
in schools" (Hoy & Miskel, 1991, p. 221). 

II. Limitations 

In our study, we consider teachers' impressions of principals' 
exercise of hierarchical independence and influence, and examine the 
extent to which these impressions affect teachers' perceptions of 
school climate. At this point, it is necessary, to acknowledge some 
important limitations of our study. District size plays a role in shaping 
the relationship between the principal and superintendent. In small 
districts, because there are fewer levels of administration to separate a 
superintendent from principals (Crowson & Morris, 1 985), the 
principal/superintendent relationship is more direct and interactions 
more frequent than in large districts. Large districts are characterized 
by many administrative levels between a principal and the 
superintendent, creating indirect and often impersonal communication 
between the two, thereby muting the relationship (Crowson & Morris, 
1985; Boyan, 1988). In order to enhance the chances of finding an 
effect, if there is one, our study occurred in a small, rural district and is 
unlikely to generalize to large or urban districts. 

Similarly, elementary schools, because they are smaller and 
structurally less complex than secondary schools, offer a better context 
for initial explorations of teachers' perceptions of the 
principal/superintendent relationship. Because our study occurred in at 
the elementary level, results may not generalize to secondary schools. 
Finally with regard to limitations, the linkage between teachers' 
perceptions of the principal/superintendent and their perceptions of 
school climate is indirect. Still, we assert that these perceptions are 
inherent in the complex reality of schools. Given the impact of school 
climate on student achievement, exploring the linkage between 
teachers' perceptions of the principal/superintendent relationship and 
school climate merits study. 

III. Statement of Problem 

The teacher perceived power vested in principals in the form of 
independence from and influence on the superintendent may relate to 
teachers' perceptions of school climate as well. Given the lack of 
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research on these phenomena, we explore the effects of hierarchical 
independence and influence on school climate as perceived by 
teachers. This case study is an illustration of how this relationship is 
played out in practice. Determining how and why teachers value 
certain aspects of their principal's relationship with the superintendent 
allows knowledge to be gained about the functioning of schools and 
how schools can become more effective. 

IV. Method 

Our study uses case analysis to compare a typical school -with a 
positive outlier, a research design recommended by effective schools 
researchers (Teddlie &, Stringfield, 1993). Results reported here are 
peirt of a larger study that took place in a southern state and 
investigated the effect of the hierarchical independence and influence 
on school climate. For the larger study, sample districts and schools 
were selected on that criterion that both the superintendent and the 
principals had been in their current position for at least 3 years. This 
criterion gave the principals and their respective superintendent a 
chance to develop a relationship before the study data were collected. 
As noted above, all participating schools were at the elementary level 
and were comprised of grades kindergarten through five. 

To gather the data, all regular education teachers in these 
schools were asked to complete two questionnaires. To measure 
teachers' perceptions of principal hierarchical independence and 
influence, the Teacher Attitude Inventory (Glascock, 1996 [TAI]), was 
developed. A panel of six experts was used to analyze possible items 
for the Teacher Attitude Inventory (TAI) survey developed. The 
experts are two professors in educational administration, one professor 
in educational research, two principals, and one 12-year veteran 
teacher. Each expert was told the purpose of the TAI survey and what 
each section is intended to measure. Modifications and changes were 
made to items based on the advice and opinions of these experts. 

The Teacher Attitude Inventory (TAI) includes 14 statements 
that measure teachers' perceptions of the principal's level of 
independence from and influence with the superintendent. 
Independence is defined as "the extent to which administrators 
demonstrate their autonomy from superiors as they interact with 
teachers" (Hoy & Miskel, 1991). This independence from and 
influence with the superintendent is measured by a five point Likert 
scale ranging from "strongly agree" to "strongly disagree", with a 
response option of "don't know" included. Scoring is completed by 
reverse coding negative questions for independence and influence, and 
the summing the seven item scores. Each set of scores is aggregated to 
the school level and the average is generated so that there is one score 
for each school. 

To assess school climate, teachers completed the Organizational 
Climate Description Questionnaire - Revised Elementary (Hoy, Tarter, 
& Kottcamp, 1991 [OCDQ-RE]). The OCDQ-RE measures school 
climate using two components. One component, principal behavior is 
comprised of three dimensions, including directive, supportive, and 
restrictive. Hoy et al. (1991) report Cronbach's alphas of .95 to .80 for 
these dimensions. The second component of the OCDQ-RE, teacher 
behavior, also consists of three components, disengaged, collegial, and 
intimate. Cronbach's alpha for these dimensions range from .90 to .75 
(Hoy et al, 1991). For each dimension in both components, scores fall 
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into one of five categories ranging from very low through average to 
yew high. 

The case studies reported here involve two schools that were 
selected using results from both questionnaires. As noted, one school 
was typical. In this school, teachers scored nearest the mean on both 
questionnaires. The other school, a positive outlier, was chosen 
because teachers scored farthest from the mean on both questionnaires. 
Serendipitously, both of these schools were located in the same 
district, permitting more meaningful comparisons. 

To gather data, both schools were visited for two days each. 
During this time, the principal and a random sample of teachers were 
interviewed. In addition, parents and ancillary personnel, such as the 
school nurse, were interviewed. A protocol explored the extent to 
which the superintendent played a role in the day-to-day life of the 
school; beliefs held by faculty and the principal about the relationship 
between the principal and superintendent; and the effect the 
relationship between the principal and superintendent had, 
respectively, on teachers and principal on a typical day. In addition, 
information was gathered. The researcher spent time observing and 
having casual interactions in the halls, cafeteria, and on the 
playground. Interviews were held for half an hour to an hour with 
individuals and taped. Classes were observed and children were 
engaged in casual conversation. The researcher kept a running record 
through tape recordings during all sessions and observations. Over 200 
pages of transcribed notes and interviews were generated. In 
presenting the results, pseudonyms are used to ensure the 
confidentiality of participating schools and individuals. 

V. Results 

The case analyses reported below explore the relationship 
between teachers' perceptions of principal hierarchical independence 
and influence and their perceptions of school climate. Findings for 
each school are presented separately, and include the results of the TAI 
(teacher perceptions of the principal's hierarchical independence and 
influence) and OCDQ-RE (teacher perceptions of school climate), a 
brief contextual description of the school and professional staff, the 
impressions of the teachers who were interviewed regarding (a) the 
principal's exercise of hierarchical independence and influence and (b) 
the school's climate. Our discussion of the results concludes with a 
comparison of the two schools in terms of teachers' perceptions of 
principal hierarchical independence and influence and their 
perceptions of school climate. 

District Description 

The two schools are in the same district as noted, hence, we 
begin with a description of the district. The Jackson County school 
district was once considered rural and poor, but now has a more 
exurban flavor and serves as a bedroom community for a nearby city. 
Over half of the population is high school graduates and nearly 1 0% 
have college degrees (U.S. Census Bureau, 1990). Of the 17,000 
students who attend the 8 schools in Jackson County, 94% are 
European American, with 6% of the students describing themselves as 
African American or other. 
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Greenbriar Elementary (typical school) 

The TAI score is 18.111 indicating that teachers have a neutral to 
positive perception of the principal/superintendent relationship. The 
Greenbriar Elementary scores in the average, low, and very low 
categories on the principal dimensions of supportive (495.258), 
directive (429.906) and restrictive (397.032) of the OCDQ. Scores are 
high in the collegial (586.357), very high in the intimate (609.299), 
and average in the disengaged (497.619) dimensions. These scores 
indicate a school in which teachers perceive moderate levels of 
principal positive or negative behaviors that impact their work life, 
teachers have good rapport with each other but teachers are somewhat 
disengaged from the workplace. This school is between the engaged 
and disengaged climates described in the typology of climate 
developed by Hoy, Tarter, and Kottcamp (1991). However, while 
teachers reported good rapport among themselves, on the typology 
developed by Hoy et al. (1991), they tended toward disengagement 
from the workplace. 

As mentioned previously, additional data were collected through 
interviews. At Greenbriar, interviews were conducted with the 
principal, 3 teachers, a student teacher, 3 parent aides, and the school 
nurse. As will be seen, these participants were uniformly positive 
about the principal. 

The school itself serves children from the working class 
community that it is located. Brickwork on the 30-year old building 
has faded to a grayish red color and the window trim is dull with age. 
Once inside the building, an orientation toward neatness and 
cleanliness is evident. Splashes of color from student artwork adorning 
classroom doors brighten the faded pastels of the hallways. 

Classrooms are clean, though many are cluttered because there is not 
enough room to hold all the instructional materials, books, art 
supplies, and maps. One teacher explained, " With so many children, 
we have no room for instructional materials." The principal's office is 
small but professional in appearance. Walls display many citations and 
awards earned by the school. Like the classrooms, the principal's desk 
is cluttered as if too many things require her attention. In the cramped 
outer office, where the secretary sits, an up-to-date computer stands 
out against the other office equipment which old and worn. 

Most of the 450 children who attend Greenbriar walk to school. 
School demographics reflect the district, with 9'4% of the students 
describing themselves as white, 6% as black, and 35% as eligible for 
free-or-reduced-price lunch. According to the Food and Nutrition 
Bureau of the state Department of Education (1995), Greenbriar is a 
low poverty school. As might be expected given the demographics, the 
students do reasonably well on standardized tests, performing above 
the district, state, and national median percentile for fourth graders on 
the California Achievement Test. 

Also consistent with the demographics, children were well 
behaved and quietly involved themselves in their assignments. In one 
of the few instances when disciplinary action was required, two 
children were sent to stand in the hallway just outside the classroom 
door. The teacher of the offending students explained they had been 
removed for talking out of turn, noting, "Children are not allowed to 
disrupt classroom activities." 

There appeared to be a genuine friendship among the teachers 
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and between the faculty and the principal. One teacher noted that, 
"Many of the teachers and the principal play Keno once a week, and 
this year, as soon as school ends, several of the teachers and the 
principal are going on vacation together." 

In addition, Greenbriar enjoyed strong parental support and 
many parent volunteers. During the 2 days of observation, 20 parents 
were observed assisting teachers in the classroom or with clerical 
chores, such as duplicating papers. These parents sincerely liked the 
teachers and the principal, and felt the school operated well. One 
commented, "The principal maintains discipline and the children know 
and follow the rules so this school runs really well." 

Curiously, of all the people interviewed, the school nurse was 
the only one who mentioned the children's intellectual growth without 
prompting from the interviewer, noting 'Ttis very important to give the 
children a chance to learn as much as possible." Neither the teachers 
nor the parent aides offered such statements. Moreover, when asked 
directly about the students' learning, teachers and parents unanimously 
responded "Discipline is very strict in the school and the principal 
does not allow children to interrupt the learning of others." The 
recurring theme that surfaced during the interviews was that discipline 
was the most important objective of the school. Asked if discipline 
was important of itself or as a prerequisite for learning, one teacher 
said "Discipline is just as important as knowledge." 

The principal, Ms. Cook, like the faculty, is a white female. 

Now in her late fifties, Ms. Cook has been an educator in the district 
for over 20 years, the last 4 of which have been as principal at 
Greenbriar. She is professional in dress and demeanor with an air of 
strength about her; yet, she smiles easily. Ms. Cook was very open to 
questions and had no hesitancy in offering opinions about teachers, 
children, the school, parents, or the district staff. She was quite proud 
of the staff rapport and mentioned the Keno nights and the upcoming 
vacation as evidence. This network of friendship was perhaps more 
important because of the contrast it offered to the prior principal. The 
prior principal was a male who the teachers viewed as authoritarian 
and unfriendly. Teachers seemed to appreciate that Ms. Cook went out 
of her way to establish good communication with them and to include 
them in group activities. Her more cordial demeanor did not hinder 
school operations, however. It was mentioned the children's 
intellectual growth without prompting from the interviewer, noting 
that "It is very important to give the children a chance to learn as much 
as possible.' Neither the teachers nor the parent aides offered such 
statements. Moreover, when asked directly about the students' 
learning, teachers and parents unanimously responded "Discipline is 
very strict in the school and tile principal does not allow children to 
interrupt the learning of others." The recurring theme that surfaced 
during the interviews was that discipline was the most important 
objective of the school. Asked if discipline was important of itself or 
as a prerequisite for learning, one teacher said, "Discipline is just as 
important as knowledge." 

As suggested above, Ms. Cook did not mention students’ 
academic or social performance until ask specifically. Even then, she 
did not speak of test scores, though it will be remembered that the 
schools scored above the state and district average median percentile, 
and when pressed about student behavior, Ms. Cook stated firmly that 
children "are here to learn. No child can interfere with the learning 
others." It was as if the children were not the purpose of the school. 
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but a separate entity, not integral to the functioning of the school. 

The principal's view of the superintendent was fairly distant. She 
did not indicate any personal relationship or friendliness, but there was 
a sense of professional respect and loyalty. Ms. Cook is one of the 
district personnel who are aligned with the superintendent, as opposed 
to the more conservative members of the school board. According to 
Ms. Cook, she supported the superintendent because he established 
good procedures, tried to respond to individual school needs, and 
addressed problems quietly. 

Ms. Cook's description of her relationship with the 
superintendent indicated limited interaction between them that might 
suggest a low level of principal influence with the superintendent. That 
type of interaction does not appear to exist between this principal and 
the superintendent. At the same time, Ms. Cook did not any negative 
feelings about 

As to disciplinary actions, very little is observed. One incident 
occurred while the researcher was in the main/secretary's office. A 
child was sent to the office during the observation period for 
disciplinary reasons. The secretary seemed very familiar with the 
student, asking him "Why are you here this time? Sit down and the 
principal will see you when she has the time." At another point, two 
children were observed standing in the hallway, next to a classroom 
door. They had been removed from their classrooms for talking out of 
turn. As one teacher explained, "Children are not allowed to disrupt 
classroom activities." 

School Climate. 

The school was visited in the late spring of the year. End of the 
year activities, such as special topics in classes where all curriculum 
needs have been met, assemblies, and parties have begun, yet there is a 
quiet, orderly feel to the school. Children are following an established 
routine, both as to activities and expected behavior. Before the first 
bell of the day, they are cheerful, talkative, and happy. Recess is 
physically active (running, climbing, and jumping) with the normal 
yelling, laughing, and small upsets that occur with children at play. 

End of the day activities are boisterous as the children become excited 
about going home. 

The teachers appear calm, their demeanor, speech and body 
language indicate quiet confidence in the overall condition of the 
school. Those teachers interviewed stated that everything is in order 
and on schedule because the children cause no extreme difficulties, the 
curriculum is being completed on time and the paper work is 
complete. 

Teachers are asked how they feel when contemplating the start 
of the day. One said, "Everything runs smoothly here, the children 
know the rules." Every teacher response given, whether about 
activities, climate or environment, expresses pleasure at the level of 
discipline in the school. It was the only response given related to the 
children. There was no bragging about the children's test scores, 
awards, and innovations in the classroom or a specific child who has 
excelled in some way. Teachers were also asked what they consider a 
good day at school. One stated that, "Any day is good when the 
children are quiet and everyone is in a good mood." 

The parent aides indicate that they are pleased with the school. 
Discipline is maintained, their efforts as aides are appreciated and even 
the one teacher who is demanding is manageable. The school is 
functioning quietly. 
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As to whether the superintendent is active in the day-to-day 
operations of the school, the teachers responded that he is not 
considered an active part of the school. This situation indicates that the 
superintendent is not a close controller of the principal's activities, as 
far as the teachers are aware. Teachers did not offer any information 
about the principal's influence with the superintendent affecting the 
day-to-day operations of the school either. No connections appeared 
evident to the teachers interviewed that the principal/superintendent 
relationship played a role in shaping the climate of the school. 

When interacting among themselves, the children laugh and talk 
with smiling faces and positive body language. In the classroom, their 
faces are mostly neutral and their bodies are slumped in their desks. 

No teacher was observed making classroom presentations with an 
excited voice or body language. The teachers appeared to be reciting 
lessons. There appears to be a missing connection between teachers 
and children, especially during class time. 

Perceived Principal/Superintendent Relationship. 

Teachers at GREENBRIAR base their opinions of the principal's 
relationship with the superintendent on two factors: resources provided 
to the school and the principal's support for the superintendent over the 
school board's conservative members. All teachers interviewed are 
aware of the discord between the superintendent and the board; they 
are aware also of the principal's loyalty to the superintendent. The 
principal has made her position clear in a staff meeting. 

The teachers feel that the school is receiving adequate resources, 
yet they are aware of the district's financial difficulties. While the 
teachers want more supplies and desire more computer equipment, 
they seem satisfied that all that could be done is being done. The 
teachers state that the principal is doing her best in trying to get more 
from the superintendent, yet seem unaware of how she is doing this 
task. As to the superintendent's role in the school, it is viewed as 
distant at best, if not non-existent. Most teachers could not remember 
if the superintendent had come to the school during the year. One 
teacher commented that, "He has more important things to do than 
come see us." 

The principal's influence on the superintendent is viewed as 
difficult to ascertain by teachers. Teachers stated that, "Getting 
supplies are the only way we can tell if the principal can move the 
superintendent." Teachers do not perceive influence in any concrete 
fashion as a form of power on the part of the principal. The ability of 
the principal to influence the superintendent is seen as too invisible to 
teachers. 

The concept of independence is easier for the teachers to 
discuss. The teachers interviewed believe that the principal is 
independent of the superintendent and makes most decisions herself. 
The teachers believe that the principal follows district guidelines and 
does not give the superintendent reasons to closely monitor the school 
or the principal. The teachers' point agrees with the research (Crowson 
& Morris, 1985) that describes principal strategies for maintaining or 
obtaining independence from the central office. Crowson and Morris 
(1985) describe principals being aware that the superintendent and 
central office will give greater latitude to principals who do not make 
trouble for the central office. This tacit understanding between 
principals and the central office is part of the informal method used to 
control district life. 
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Summary. 

Greenbriar is an elementary school in a poor district, which has 
financial problems and leadership tensions as well. While there is a 
general awareness of the leadership tension, the awareness does not 
appear to cause great stress in the teachers or the school in general. 

The teachers appear to, be able to isolate themselves from the district 
level tensions. 

The school functions in a traditional manner, with discipline 
being the main objective of teachers, parents and principal. There is 
consensus among the adults that the school runs well. The teachers' 
comments appear to show a concentration on creating a pleasant work 
environment for themselves. The present principal has not been in 
place for a lengthy tenure so it would be of interest to follow this 
school and see if any changes develop which might demonstrate any 
awakening to potential problems. 

Teachers appear to value their principal for sharing the same 
outlook on the mission of the school that is discipline above all else. 
The teachers also value personal friendships with the principal. There 
does not appear to be any overt awareness on the part of teachers about 
the principal/superintendent relationship. 

Teachers do not indicate any belief that the principal is 
influential with the superintendent, rather the teachers appear to have 
no perceptions about that aspect of the relationship. The only 
indication that even hints at influence is that teachers feel an adequate 
amount of resources are available for the school. Independence is also 
rather vague for these teachers but somewhat stronger than influence. 
The teachers appear to believe that because there is no evidence of 
interference in school activities by the superintendent, the principal 
must be independent of the superintendent to a high degree. As with 
influence, the teachers show only vague interest or awareness in the 
principal/superintendent relationship. Rather, teachers appear to be 
centered on the relationships among themselves and with the principal 
only. The possibility of independence and influence being interwoven 
is not apparent. 

Waterfall Elementary (high scoring school) 

The TAI score is 23.737 indicating that the teachers have a 
positive perception of the principal/superintendent relationship On 
iheOCDQ, the high elementary school Waterfall scores are in the very 
high category on the principal dimension of supportive (699.814), 
average in the directive dimension (510.938), and very low in the 
restrictive dimension (306.581). Scores are very high in the collegial 
(673.457) and intimate (672.570) dimensions and very low in the 
disengaged (355.079) dimension. This indicates a school in which 
teachers perceive high levels of principal positive behaviors and low 
levels of negative behavior which impacts teacher work life; teachers 
have good rapport with each other and are actively engaged in their 
work. 

Waterfall is an elementary school, serving grades K-5 plus 
special education classes. There are 383 children and 26 faculty 
members. All the children are white as are the teachers (AFSR, 1994). 
Waterfall has 27.5% of their classes in the 1-20 range and the rest of 
the classes in the 21-26 range. Student attendance (96%) is also better 
than the district average (95.45%). 
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CRT results show the children to be scoring higher than the 
district and state average. Waterfall places fourth out of 1 8 district 
elementary schools on the CRT for grade three; and places fifth out of 
1 8 district elementary schools on the CRT for grade five. CAT results 
for the fourth grade place fourth in the district in overall performance 
and well above the district, the state, and the national median 
percentile (PPDCR, 1994). 

Waterfall is located in the rural, southern part of the district. The 
school is located in a curve of a secondary country road. Across the 
road from the school is a small hardware store. There are no 
neighborhoods, houses or commercial entities (other than the one 
mentioned) within half of a mile of the school. This part of the district 
consists of citizens who are considered to be poorer than the people in 
the northern two thirds of the district. Most of the population in the 
southern third is considered transient. 

Waterfall is over thirty years old., simple in design and consists 
of two faded red brick buildings and a modular cafeteria. Since the 
building is on a curve of a secondary, rural road that has been the sight 
of several accidents, attempts have been made to reinforce the chain 
link fence surrounding the property. The front drive where buses and 
cars dropped off children has a courtyard appeal with three large trees 
shadowing the pavement and the front of the buildings. Artwork dots 
the classroom windowpanes. 

The interior of the school is spotless. The janitor starts polishing 
the floor as soon as the children begin the first class. Children's 
artwork is arranged beside the classroom doors and the colors of the 
walls are pastel and cool to the eye. The school is well lit and not 
cluttered with boxes, supplies or equipment. The cafeteria is spotless 
with the chairs and tables wiped clean and ready for children. There is 
new equipment purchased recently visible in the kitchen area. 

The classrooms have neat cupboards to store supplies and there 
appears to be plenty of space for the children to move comfortably. 

The desks are widely spaced and there appears to be plenty of room 
the children to move around comfortably. The building is relatively 
old but inside the classrooms the age does not seem apparent. Instead 
the materials and furniture appear sturdy, up to date and useful. 

The children are transported to school by bus and car. The 
children are from agricultural and working class families with incomes 
averaging below $30,000 (U.S. Census Bureau). Most of the children 
are dressed in clean but faded clothes with some of the clothes being 
too big or small. Thirty-four percent of the children qualified for the 
Breakfast program (Food and Nutrition Bureau, Louisiana Department 
of Education) putting Waterfall in the moderately poor category of 
schools. It should be noted that although there is a category break 
between the two schools, only one percentage point separates the 
schools in the measure of poverty. 

Morning activities are boisterous (laughing and socializing 
loudly) and there are many interactions among teachers and children. 
These interactions include socializing and movement around the 
hallways, asking questions of teachers, receiving reassurance from 
teachers as to the day's activity. The children are observed quietly 
working in classes, helping each other and asking questions of their 
teachers. The children have smiles and positive body language such as 
sitting up straight, leaning forward and actively seeking inclusion by 
raising their hands. 

All 26 teachers are white females who are local residents except 
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for four from a nearby city who drive in each day. Thirty-five percent 
of the teachers hold at least a Master's degree and one holds a Ph.D. 
This percentage places the school above the district average but below 
the state average (PPDCR, 1994). The principal strongly encourages, 
both verbally and with financial support, those teachers who wish to 
pursue professional development activities. Teachers are taking 
advantage of these opportunities. 

Two-day visit, five teachers are interviewed. Two themes 
emerge in these discussions. One theme is that children are the first 
priority of teachers and the principal. Each teacher interviewed 
mentioned the children, learning and the social needs of the children as 
their main concern and interest. The second theme is the teachers' 
enthusiasm for working with children. The teachers are enthusiastic 
about the children, future professional development, the principal's 
role in the school and each other. One teacher. Teacher A, who moved 
to the school from a large, urban school district in Texas, is most 
enthusiastic. Teacher A readily shared her ideas about the children, the 
teachers and the principal. Teacher A feels the school functions well 
because "the principal's personality and friendliness directly influence 
the attitudes of both teachers and children." 

Another interviewee, a third grade teacher. Teacher B, expressed 
her concerns about the children, stating, "These children often come 
from broken homes and many times they don't know where they will 
be sleeping that night. This school is their only stability." Teacher B is 
most concerned about the children as people and feels the atmosphere 
of the school helps these children cope with the stress they have at 
home. Teacher B stated that "children need a nurturing presence in 
their lives and unfortunately we are the only ones who give it to them 
sometimes". 

A kindergarten teacher stated that the principal's willingness to 
approach the superintendent about new techniques for "hands on 
manipulatives" has improved the learning experience of her students. 
This kindergarten teacher feels that the principal has brought many 
new ideas to the school during the five years she has been there and 
also appreciates the principal's continued support for workshops and 
seminars. "The workshops are very important and when we go out of 
town for one, we try to save the money for travel so we can spend it on 
supplies for the children," she explained. 

The principal is a white female in her late forties or early fifties. 
The principal had been a teacher in the district for almost twenty years 
before moving to WATERFALL three years previously. She is very 
open and friendly, offering to assist the researcher in any possible way. 
No visible sign of stress are evident in the teachers at these 
interruptions. The teachers act as if the interruption were a normal 
occurrence. The teachers have been prepared for the arrival of the 
researcher. The principal has given all the staff nametags, telling the 
researcher it will make the process friendlier. 

The principal expressed pride about the school in three ways. 
First, the children are cared for both emotionally and physically and a 
good learning environment is provided for them. Second, the principal 
is proud of her teachers because they are willing to improve their 
professional skills and are actively pursuing further educational 
opportunities. The principal stated "I try to provide funds for any 
teacher who wants to attend workshops that improve their teaching and 
bring new techniques to the classroom." Third, the principal is 
impressed by the community dedication to the school. An example of 
that dedication is the group of five women who run the library for the 
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school. Not one of the five women has a child in the school, but they 
want to perform this service for their community. The principal stated 
"This community ownership of the school rubs off on the attitude of 
teachers and students alike." 

The principal is also proud of the instructional choices made by 
the school staff. In the previous year, the principal offered the teachers 
a choice of either a librarian or physical education teacher for a new 
staff position. The teachers decided that the five women running the 
library performed well so they chose the physical education teacher. 
Two things are significant about this event. First, the principal did not 
make the choice, she allowed the teachers to make the choice. Second, 
the reason for the physical education choice was predicated on the 
rather poor physical condition and coordination of many of the 
children. In other words, the teachers make choices based on the 
immediate needs of the children. As the principal described it, 
"Teachers in this school care deeply for the children's well being, both 
mental and physical." 

This principal has a professional and personal relationship with 
the superintendent. She taught his children and knows the 
superintendent as a parent. The principal is very active in the district, 
serving as president of the district principal's association and working 
closely with the central office to find resources for her school. The 
principal said, "I bother them to death and they give me some of what I 
want just to get rid of me." 

School Climate 

The school was visited in the spring of the year. The hallways 
are quiet and everyone, teachers and children alike, are busily engaged 
in learning activities. No discipline problems are noticed. A general 
atmosphere of cooperation exists. 

When teachers were asked how they approach each day, one 
responded that, "coming to school is fiin, I really get excited about 
seeing the children." All the comments were positive. The teachers 
feel relaxed and comfortable with their school. When asked what a 
good day at school is like, teachers offer smiles and say "A good day is 
when everyone learns." "When the children come to school and can 
learn because they are not hungry and home was quiet the night 
before." "When there is laughter and we get that a lot here." 

One teacher offered her curriculum choices as an example of 
how the principal allows creativity in the classroom. At the end of the 
spring term, this teacher works with the children on a crafts approach 
to Louisiana culture. "I have the children make different types of maps 
using beans native to the state for materials. I bring in Cajun 
storytellers, singers, and even a politician or two to speak with the 
children." Her classroom hums along and buzzes with activities. The 
teacher moves from one group of children to another, offering 
comments and answering questions. As she tells the story, she smiles 
and her eyes twinkles, "The children can laugh, talk, and learn all at 
the same time." 

WATERFALL teachers hold the same views as GREENBRIAR 
teachers about the superintendent. The teachers really only perceive 
the superintendent through the principal. The principal is perceived as 
funneling the superintendent input to the school and because the 
teachers feel supported in their efforts, the superintendent is perceived 
as having an indirectly positive effect on the school. The principal is 
perceived by the teachers as having a good relationship with the 
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superintendent, both formal and informal. The formal relationship is 
demonstrated through the principal’s high profile with the central 
office and her ability to receive what the teachers perceive as special 
attention for the school. The teachers believe that this situation is 
another demonstration of the positive nature of their school. 

Three themes emerge about climate based on observations and 
interviews. First, children are the focus of school activities and the 
principal and teachers work to give the school a conducive learning 
atmosphere. Second, the principal succeeds in building an air of 
professional energy about teacher growth because growth is viewed as 
important to enhance the abilities of teachers. Third, the principal 
creates an aura of efficiency and effectiveness by providing resources 
for the children and teachers. The teachers perceive their principal as 
having positive influence with the superintendent which has proven 
beneficial for their school by providing resources, including funds for 
professional growth and classroom needs such as the computer lab. 

The teachers also believe that positive results have grown from the 
principal's ability to act independently of the superintendent. This 
independence has been displayed by the principal being allowed to 
incorporate new curriculum designs in the classroom that are not 
necessarily in line with district policy. The teachers believe that the 
principal was able to accomplish this task because the superintendent 
gave her greater independence because of the superintendent's trust in 
the principal's abilities. 

Perceived Principal/Superintendent Relationship 

Teacher responses at WATERFALL demonstrate only vague 
awareness of the principal's relationship with the superintendent. The 
teachers know of the principal's previous history with the 
superintendent and that she is able to speak with the superintendent 
more often than would be generally expected. Teachers also know how 
hard the principal works to gain resources from the central office. 
Unlike GREENBRIAR teachers, these teachers do not mention the 
antagonism between the school board and the superintendent. Either it 
is removed from their immediate focus or the teachers do not think it 
appropriate for discussion. 

Teachers explain that their school is receiving more resources 
than other schools in the district because of the efforts of the principal. 
One stated, "The principal is constantly thinking of new ways to move 
the superintendent toward new curriculum and innovative programs." 

A kindergarten teacher is particularly vocal on this issue. She said, 
"Without the principal's support I would never have gone to the 
workshops and learned about new ways to stimulate my slow learner." 
As stated in relationship to climate, the teachers feel that the principal 
is able to have great influence with the superintendent. The principal is 
also able to act independently because the superintendent trusts her 
judgment. For example, the principal is allowed to modify curriculum 
in the school rather than strictly follow district policy, as mention in 
the climate section. 

As to a role for the superintendent in the school, the teachers do 
not see it as direct. His role, as explained by one teacher, "is to manage 
finances, talk to the board and provide the schools what they need." 
The superintendent's role is viewed as being indirect and funneled 
through the principal. The principal is the link between the school and 
the outside world. As with GREENBRIAR teachers, WATERFALL 
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teachers are vague about many aspects of the principal/superintendent 
relationship. 

However, WATERFALL teachers appear to view their principal 
as being very influential with the superintendent. The personal nature 
of the relationship is given as one reason for this success (Hart, 1993). 
WATERFALL teachers appear to be aware of and value the principal's 
hierarchical independence and influence. The teachers believe that the 
principal's ability to act independently is a sign of influence with the 
superintendent. The principal's independence is valued as a resource 
by the WATERFALL teachers. 

Summary 

WATERFALL functions in a participative type of 
administration. The WATERFALL principal gives the teachers a great 
deal of autonomy and allows teachers to participate in school wide 
decisions. There is consensus among the WATERFALL teachers that 
the school is working well. One teacher said, "Things run smoothly 
here and the children are learning." Based on interviews, there is a 
united goal for WATERFALL and that goal is to help the children 
learn. Attaining this goal is being accomplished in three ways: teachers 
and the principal work to create a positive learning environment for 
the children; teachers are being encouraged to grow professionally for 
their personal benefit and the benefit of the children; and resources are 
found by the principal to enhance the learning environment. 

This much can be deduced from the teachers remarks, the 
principal proves to be influential by providing resources for the benefit 
of the school. The principal is also able to act independently about 
such matters as curriculum. The principal is able to give the teachers 
latitude to try new curriculum approaches. The teachers view this as 
independence on the part of the principal as beneficial to the school. 
From the teachers' viewpoint both hierarchical independence and 
influence are perceived within the principal/superintendent 
relationship. 

Comparison of Greenbriar and Waterfall Related to 
Principal/Superintendent Relationship and School Climate 
as Perceived by the Teachers 

When comparing schools, it is important to ask the same types 
of questions and look for the same types of situations and information. 
True comparisons can then be made. Nuances and specific differences 
are discovered and play an important role. While no attempt was made 
to choose schools in the same district, the situation occurred and 
allowed the researcher to make more in depth comparisons since the 
schools share the same superintendent. There are differences in the 
OCDQ dimension and TAI scores (see Table 1) that offer a beginning 
point for a discussion of the two schools. The qualitative section on the 
present research offers greater insight into the differences recognized 
by Xh^OCOQ and the TAI. 



Table 1 

Case Studies: Comparison of OCDQ and 7)4/ Scores, 
Demographic Information, and Academic Tests Results 
Between the Greenbriar and Waterfall Elementary 

Schools 
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Item 


Greenbriar Waterfall 


OCDQ dimension scores 




Supportive 


495.3 


699.8 


Directive 


429.9 


510.9 


Restrictive 


397.0 


306.6 


Collegial 


586.4 


673.5 


Intimate 


609.3 


672.6 


Disengaged 


497.6 


355.1 


TAI 


18.1 


23.7 


Student Population 


450 


383 


Faculty Size 


32 


26 


Student Attendance 


95.73 


96% 


Suspensions 


4.43% 


.69% 


CRT-Grade 3 (Language/Mathematics) 


96/97 


98/100 


CRT Scores-Grade 5 
(Language/Mathematics) 


84/91 


94/98 


CAT Scores-Grade 4 
Median Percentile 


69.5 


72.7 



Principal 

Principals at the two schools differ in many ways. The two most 
important deal with the mission of the school and norms for the 
professional level of teachers. At GREENBRIAR, the mission is to 
maintain a quiet, well-disciplined student body that does not disrupt 
the teachers' work environment. The principal said, "No child is 
allowed to disrupt class." While at WATERFALL, the mission is to 
provide a nurturing learning environment for the children. This 
difference between the two principals' results in a teacher centered 
environment at GREENBRIAR and a child-centered environment at 
WATERFALL. 

The second difference deals with the professional development 
of teachers. The principal or teachers at GREENBRIAR did not 
mention professional development and when asked, the principal 
replied, "the school year is too busy for the teachers already so I leave 
that decision up to individual teachers". At WATERFALL, the 
principal finds seminars, workshops and other opportunities for the 
teachers to grow professionally and publicizes these events to her 
teachers. The WATERFALL principal actively recruits teachers to 
attend the events and finds incentives, both financial (district and 
private) and emotional, to entice the teachers. 

Another difference is worth noting. The WATERFALL 
principal has good rapport with the superintendent that appears to give 
her greater access and a greater willingness to interact with the central 
office for obtaining resources. This personal relationship between the 
principal and the superintendent appears to be a significant contributor 
to the principal's success in obtaining resources. 
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School Climate 

In both schools, the teachers state that they are satisfied with the 
psychological feel of their school, yet there are telling differences. 
GREENBRIAR teachers appear satisfied with the status quo while 
WATERFALL teachers are eager to use new ideas and approaches to 
education. GREENBRIAR appears to be ruled by a need for discipline 
while WATERFALL appears to be ruled by the children's needs. It is 
as if GREENBRIAR principal, teachers and parents view the children 
only in terms of how the children affect the adults in the school. On 
the other hand, WATERFALL appears to be concerned with the 
children's needs before any others. At WATERFALL discipline is one 
of many tools helping to create a good learning environment for the 
children. Resources, new ideas, innovations in curriculum choices and 
nurturing of the whole child are just as important tools. 

Perceived Principal/Superintendent Relationship 

Neither GREENBRIAR nor WATERFALL teachers state any 
knowledge about the principal/ superintendent relationship beyond the 
principal's ability to garner resources for their school and act 
independently. GREENBRIAR teachers are aware of their principal's 
support for the superintendent over the school board but appear 
unaffected by the situation. WATERFALL teachers are aware of their 
principal's personal relationship with the superintendent and value that 
relationship in terms of influence (resource allocations) and 
independence (curriculum changes). 

This finding moves in concert with those items on the Lf/that 
deal with influence as the principal's ability to gather resources for the 
benefit the school. At the same time, the WATERFALL principal 
demonstrates independence in decision making about curriculum 
materials and new instructional approaches. WATERFALL teachers 
perceive both hierarchical independence and influence together. 

VI. Conclusion 

One theme emerges from these two case studies. 
GREENBRIAR and WATERFALL staffs view the missions of their 
school differently. At GREENBRIAR, the staff expresses their 
mission as a concern for their work environment and that the school 
maintains "good discipline." The mission at WATERFALL is to 
provide a good learning enviroiunent for the children. At 
GREENBRIAR learning is secondary to discipline and at 
WATERFALL learning is the number one priority. Climate is 
perceived differently because of the GREENBRIAR teachers differ in 
what they value from the WATERFALL teachers. The WATERFALL 
teachers value a climate that is conducive to learning and nurturing 
children; GREENBRIAR teachers value a climate that is conducive to 
a smooth working situation for them. 

Specific to the present study, indications of hierarchical 
independence and influence are very different at the two schools. At 
GREENBRIAR, teachers appear to be unaware of any direct 
relationship between their principal and the superintendent, other than 
the formal, organizational relationship. GREENBRIAR teachers do 
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not voice any awareness of principal influence with the superintendent, 
other than receiving necessary resources at an adequate level. The 
principal at GREENBRIAR is considered to be in charge of the school 
and the superintendent is not seen as an interruptive force that 
indicates some independence on the part of the principal. That 
independence is not overtly displayed though; rather the teachers 
almost view the school as an island that is semi-detached from the rest 
of the world. The display of independence is an almost passive, caused 
more by circumstances than by action. 

The climate of GREENBRIAR is good for teachers and 
principals, in their view. But that view is very narrow, as if the 
teachers and principal are wearing blinders. The teachers and principal 
do not consider the children in their scope of understanding what their 
school is. If the teachers are happy in their personal relationships with 
each other and the principal, then all must be well, according to the 
teachers. This finding places the teachers' perceptions of the 
principal/superintendent relationship and organizational climate in 
perspective. The principal/superintendent relationship and climate, 
when viewed in isolation, offer evidence about their possible 
connection. These two elements are not the only elements needed to 
form a good learning environment for children. Bossert (1982) and 
Duckworth (1984) both place the principal and climate in mediating 
positions, not major causality positions for student learning. 

The missing component is the children. This development 
showcases the limitations of research that does not include the 
perceptions of all organizational groups. By leaving the children out, 
the research is limited in the ability to fully understand the true nature 
of the school climate. The teachers do believe that the principal 
directly affects the climate of their school. This is demonstrated 
through the comments about smooth operations and lack of problems 
with discipline at the school. The problem is not so much that the 
climate is disengaged or closed, rather the problems is at the mission 
level of organizations. These teachers and the principal are not 
motivated by children's needs, rather they are motivated by personal 
needs and there is no apparent dissatisfaction with that situation. 

As to connections between hierarchical independence and 
influence and climate, the GREENBRIAR teachers appear to value the 
principal's independence and link it to the smooth running of the 
school. The smooth running appears is the GREENBRIAR teachers' 
perception of their school climate. The GREENBRIAR teachers are 
happy in their workplace and GREENBRIAR teachers have positive 
feelings about their school climate. This finding does not agree with 
the results of the quantitative study. 

WATERFALL is a very different situation. WATERFALL 
teachers view their principal's relationship with the superintendent as 
dynamic, personal, and professional. WATERFALL teachers view the 
principal as actively seeking both independence and influence. 
Independence is represented through curriculum changes that are not 
in keeping with district requirements. Obtaining resources demonstrate 
influence in abundance in the teachers' view. The WATERFALL 
teachers believe that their school gets more materials and equipment 
and receive these resources quicker than other schools in the district. 
WATERFALL teachers attribute this situation directly to their 
principal's dynamic and multifaceted relationship with the 
superintendent. The relationship is seen as both personal and 
professional since the principal taught the superintendent's children. 
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The climate at WATERFALL is open, dynamic, and energetic. 
Teachers and principal alike are motivated by the children's needs. 
WATERFALL teachers actively examine new methods of instruction, 
new curricula, new resource materials, and share with each other the 
children's successes. The WATERFALL principal actively encourages 
the professional growth of her teachers, both financially and 
emotionally. The WATERFALL teachers view the principal/ 
superintendent relationship as being one of the primary reasons for the 
principal's success in improving the school and creating the school's 
positive climate. 
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